Merge lp:~milo/linaro-ci-dashboard/lava-data-model into lp:linaro-ci-dashboard

[Milo Casagrande]

Merge proposal to keep track and discuss the LAVA model and other implementation details.

Discussin with Danilo we thought about creating a separate app for the ci-dashboard, more or less as jenkinsserver.
In this first implmentation there is the structure of the app, initial implementation for the model, and a lib subdirectory ofr a library to interact with LAVA XML-RPC.

[lp:~milo/linaro-ci-dashboard/lava-data-model]

53. By Milo Casagrande on 2012-09-19

Merged from trunk.

[Stevan Radaković (stevanr)]

This looks OK for now.
I guess you already have these things in mind, but I'm going to list additional things to be extended here:
1. Extend the test status to match all available status results as in LAVA and move the status choices list to i.e. settings so it can be used in both loop_build and lavatest classes.
2. Add the unit tests for lava library.
3. I noticed in LAVA that each test contain multiple entries (i don't know what entity is this you can see in https://validation.linaro.org/lava-server/dashboard/tests/lava/ ) which that have Total results and total failures and stuff like that. So how do we plan to cover this, or do we want to do it at all?

[Milo Casagrande (milo)]

On Wed, Sep 19, 2012 at 3:46 PM, Stevan Radaković
<email address hidden> wrote:
> This looks OK for now.
> I guess you already have these things in mind, but I'm going to list additional things to be extended here:
> 1. Extend the test status to match all available status results as in LAVA and move the status choices list to i.e. settings so it can be used in both loop_build and lavatest classes.

Yeah, I was thinking where to move those values, and settings.py is a
good place to store the all for both of them.

> 2. Add the unit tests for lava library.

Sure, that is way needed. Didn't insert it at the moment, need to
clear some things out first.

> 3. I noticed in LAVA that each test contain multiple entries (i don't know what entity is this you can see in https://validation.linaro.org/lava-server/dashboard/tests/lava/ ) which that have Total results and total failures and stuff like that. So how do we plan to cover this, or do we want to do it at all?

I think we might not cover that at the moment. That is the total
number of runs for that test. Indipendently of the
board/build/whatever. It is just a big total of how many times that
test has been used, not really relevant to what we are doing.

The other thing I'm looking into, is how to best retrieve information
out from LAVA, even server side in an asynchronous way, without having
to deal with cron job. I was thinking about using RabbitMQ/Celery for
this: schedule the necessary tasks, and let the queue handle them. But
those are two new dependencies we have to deal with (maybe good to
start discussion on the mailing list).

Thanks for taking a look!

--
Milo Casagrande
Infrastructure Engineer
Linaro.org <www.linaro.org> │ Open source software for ARM SoCs

[lp:~milo/linaro-ci-dashboard/lava-data-model]

54. By Milo Casagrande on 2012-09-20

Added initial tests selection view.

55. By Milo Casagrande on 2012-09-20

Added field to store list of tests for each loop, needs custom widget.

56. By Milo Casagrande on 2012-09-21

Added LAVA widget for test selection.

57. By Milo Casagrande on 2012-09-21

Merged from trunk.

58. By Milo Casagrande on 2012-09-21

Added OK button to dialog.

59. By Milo Casagrande on 2012-09-21

Clean up and fixes.

60. By Milo Casagrande on 2012-09-21

Added hard-coded test names.

61. By Milo Casagrande on 2012-09-21

Added lavalib tests, started porting of linaro-android-tools.

62. By Milo Casagrande on 2012-09-21

Added placeholder for new method.

=== modified file 'Makefile'
--- Makefile	2012-09-14 13:44:53 +0000
+++ Makefile	2012-09-21 15:30:40 +0000
@@ -12,6 +12,8 @@
 			echo "No changes to frontend models."
 	python dashboard/manage.py schemamigration jenkinsserver --auto || \
 			echo "No changes to jenkins models."
+	python dashboard/manage.py schemamigration lava --auto || \
+			echo "No changes to lava models."
 	python dashboard/manage.py schemamigration android_build --auto || \
 			echo "No changes to android_build models."
 	python dashboard/manage.py schemamigration kernel_build --auto || \
 
=== modified file 'dashboard/frontend/forms/loop_form.py'
--- dashboard/frontend/forms/loop_form.py	2012-09-10 14:27:16 +0000
+++ dashboard/frontend/forms/loop_form.py	2012-09-21 15:30:40 +0000
@@ -23,6 +23,7 @@
     )
 from frontend.models.loop import Loop
 from frontend.widgets.chain_loop_widget import ChainLoopWidget
+from lava.widgets.lava_test_select_widget import LavaTestSelectWidget
 
 
 class LoopForm(ModelForm):
@@ -31,18 +32,20 @@
     """
     class Meta:
         model = Loop
-        exclude = ('server')
+        exclude = ('server', 'lava_tests')
 
     next_loop = IntegerField(widget=ChainLoopWidget, required=False)
     type = CharField(widget=HiddenInput, initial=Loop.__name__,
                      required=False)
+    lava_tests = CharField(widget=LavaTestSelectWidget, required=False)
 
     def save(self, *args, **kwargs):
         instance = super(LoopForm, self).save(commit=False)
-        if self.data["next_loop_0"]:
+        if self.data.get("next_loop_0", None):
             next_loop = Loop.objects.get(id=self.data["next_loop_0"])
             instance.next_loop = next_loop
-
+        if self.data.get("lava_tests_0", None):
+            instance.lava_tests = self.data["lava_tests_0"]
         instance.save()
 
         return instance
 
=== added file 'dashboard/frontend/migrations/0007_auto__add_field_loopbuild_lava_test_status.py'
--- dashboard/frontend/migrations/0007_auto__add_field_loopbuild_lava_test_status.py	1970-01-01 00:00:00 +0000
+++ dashboard/frontend/migrations/0007_auto__add_field_loopbuild_lava_test_status.py	2012-09-21 15:30:40 +0000
@@ -0,0 +1,52 @@
+# encoding: utf-8
+import datetime
+from south.db import db
+from south.v2 import SchemaMigration
+from django.db import models
+
+class Migration(SchemaMigration):
+
+    def forwards(self, orm):
+        
+        # Adding field 'LoopBuild.lava_test_status'
+        db.add_column('frontend_loopbuild', 'lava_test_status', self.gf('django.db.models.fields.CharField')(max_length=20, null=True, blank=True), keep_default=False)
+
+
+    def backwards(self, orm):
+        
+        # Deleting field 'LoopBuild.lava_test_status'
+        db.delete_column('frontend_loopbuild', 'lava_test_status')
+
+
+    models = {
+        'frontend.loop': {
+            'Meta': {'object_name': 'Loop'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'is_official': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'is_restricted': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '200'}),
+            'next_loop': ('django.db.models.fields.related.OneToOneField', [], {'blank': 'True', 'related_name': "'previous_loop'", 'unique': 'True', 'null': 'True', 'to': "orm['frontend.Loop']"}),
+            'server': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': "orm['jenkinsserver.JenkinsServer']", 'null': 'True'}),
+            'type': ('django.db.models.fields.CharField', [], {'max_length': '50', 'blank': 'True'})
+        },
+        'frontend.loopbuild': {
+            'Meta': {'ordering': "['-id']", 'unique_together': "(('build_number', 'loop'),)", 'object_name': 'LoopBuild'},
+            'build_number': ('django.db.models.fields.IntegerField', [], {'default': 'None'}),
+            'duration': ('django.db.models.fields.DecimalField', [], {'max_digits': '8', 'decimal_places': '2'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'lava_test_status': ('django.db.models.fields.CharField', [], {'max_length': '20', 'null': 'True', 'blank': 'True'}),
+            'loop': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['frontend.Loop']"}),
+            'remote_number': ('django.db.models.fields.IntegerField', [], {'null': 'True', 'blank': 'True'}),
+            'result_xml': ('django.db.models.fields.TextField', [], {'null': 'True', 'blank': 'True'}),
+            'status': ('django.db.models.fields.CharField', [], {'max_length': '20'})
+        },
+        'jenkinsserver.jenkinsserver': {
+            'Meta': {'object_name': 'JenkinsServer'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'password': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'url': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
+            'username': ('django.db.models.fields.CharField', [], {'max_length': '100'})
+        }
+    }
+
+    complete_apps = ['frontend']
 
=== added file 'dashboard/frontend/migrations/0008_auto__add_field_loop_lava_tests.py'
--- dashboard/frontend/migrations/0008_auto__add_field_loop_lava_tests.py	1970-01-01 00:00:00 +0000
+++ dashboard/frontend/migrations/0008_auto__add_field_loop_lava_tests.py	2012-09-21 15:30:40 +0000
@@ -0,0 +1,53 @@
+# encoding: utf-8
+import datetime
+from south.db import db
+from south.v2 import SchemaMigration
+from django.db import models
+
+class Migration(SchemaMigration):
+
+    def forwards(self, orm):
+        
+        # Adding field 'Loop.lava_tests'
+        db.add_column('frontend_loop', 'lava_tests', self.gf('django.db.models.fields.CharField')(max_length=255, null=True, blank=True), keep_default=False)
+
+
+    def backwards(self, orm):
+        
+        # Deleting field 'Loop.lava_tests'
+        db.delete_column('frontend_loop', 'lava_tests')
+
+
+    models = {
+        'frontend.loop': {
+            'Meta': {'object_name': 'Loop'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'is_official': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'is_restricted': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'lava_tests': ('django.db.models.fields.CharField', [], {'max_length': '255', 'null': 'True', 'blank': 'True'}),
+            'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '200'}),
+            'next_loop': ('django.db.models.fields.related.OneToOneField', [], {'blank': 'True', 'related_name': "'previous_loop'", 'unique': 'True', 'null': 'True', 'to': "orm['frontend.Loop']"}),
+            'server': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': "orm['jenkinsserver.JenkinsServer']", 'null': 'True'}),
+            'type': ('django.db.models.fields.CharField', [], {'max_length': '50', 'blank': 'True'})
+        },
+        'frontend.loopbuild': {
+            'Meta': {'ordering': "['-id']", 'unique_together': "(('build_number', 'loop'),)", 'object_name': 'LoopBuild'},
+            'build_number': ('django.db.models.fields.IntegerField', [], {'default': 'None'}),
+            'duration': ('django.db.models.fields.DecimalField', [], {'max_digits': '8', 'decimal_places': '2'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'lava_test_status': ('django.db.models.fields.CharField', [], {'max_length': '20', 'null': 'True', 'blank': 'True'}),
+            'loop': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['frontend.Loop']"}),
+            'remote_number': ('django.db.models.fields.IntegerField', [], {'null': 'True', 'blank': 'True'}),
+            'result_xml': ('django.db.models.fields.TextField', [], {'null': 'True', 'blank': 'True'}),
+            'status': ('django.db.models.fields.CharField', [], {'max_length': '20'})
+        },
+        'jenkinsserver.jenkinsserver': {
+            'Meta': {'object_name': 'JenkinsServer'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'password': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'url': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
+            'username': ('django.db.models.fields.CharField', [], {'max_length': '100'})
+        }
+    }
+
+    complete_apps = ['frontend']
 
=== modified file 'dashboard/frontend/models/loop.py'
--- dashboard/frontend/models/loop.py	2012-09-14 11:36:54 +0000
+++ dashboard/frontend/models/loop.py	2012-09-21 15:30:40 +0000
@@ -40,6 +40,7 @@
                                      unique=True, on_delete=models.SET_NULL,
                                      related_name='previous_loop',
                                      verbose_name='Next loop')
+    lava_tests = models.CharField(max_length=255, blank=True, null=True)
 
     def __init__(self, *args, **kwargs):
         self.log = Logger.getClassLogger(self)
 
=== modified file 'dashboard/frontend/models/loop_build.py'
--- dashboard/frontend/models/loop_build.py	2012-09-13 13:56:36 +0000
+++ dashboard/frontend/models/loop_build.py	2012-09-21 15:30:40 +0000
@@ -40,6 +40,7 @@
         (SUCCESS, 'SUCCESS'),
         (ABORTED, 'ABORTED'),
     )
+
     FINISHED_STATUSES = [FAILURE, SUCCESS]
     NON_FINISHED_STATUSES = [SCHEDULED, RUNNING]
 
@@ -49,6 +50,8 @@
     status = models.CharField(max_length=20, choices=BUILD_STATUS)
     duration = models.DecimalField(max_digits=8, decimal_places=2)
     result_xml = models.TextField(null=True, blank=True)
+    # General status of the LAVA tests for this build.
+    lava_test_status = models.CharField(max_length=20, blank=True, null=True)
 
     def save(self, *args, **kwargs):
         loop_builds = LoopBuild.objects.filter(loop=self.loop).order_by("-id")
 
=== modified file 'dashboard/frontend/templates/loop_detail.html'
--- dashboard/frontend/templates/loop_detail.html	2012-09-13 09:13:00 +0000
+++ dashboard/frontend/templates/loop_detail.html	2012-09-21 15:30:40 +0000
@@ -21,8 +21,8 @@
     </div>
     {% block values %}
             {% comment %}
-            This is used in the textfield detail views, to give a spece
-            to represent the key<>value values inserte by the user in a better
+            This is used in the textfield detail views, to give a space
+            to represent the key<>value values inserted by the user in a better
             way.
             {% endcomment %}
     {% endblock values %}
 
=== added directory 'dashboard/img'
=== added file 'dashboard/img/loading.gif'
 Binary files dashboard/img/loading.gif	1970-01-01 00:00:00 +0000 and dashboard/img/loading.gif	2012-09-21 15:30:40 +0000 differ
=== added directory 'dashboard/js/lava'
=== added file 'dashboard/js/lava/lava_test_select.js'
--- dashboard/js/lava/lava_test_select.js	1970-01-01 00:00:00 +0000
+++ dashboard/js/lava/lava_test_select.js	2012-09-21 15:30:40 +0000
@@ -0,0 +1,28 @@
+$().ready(function () {
+    "use strict";
+    init_lava_dialog();
+    $('#lava_select_link').click(function () {
+        populate_dialog();
+        $('#lava_select_dialog').dialog('open');
+    });
+});
+
+function init_lava_dialog() {
+    "use strict";
+    $('<div id="lava_select_dialog"></div>').dialog({
+        autoOpen: false,
+        title: 'Select LAVA Tests',
+        draggable: false,
+        height: 480,
+        width: 640,
+        modal: true,
+        resizable: false
+    });
+}
+
+function populate_dialog() {
+    "use strict";
+    $.get('/lava/select/', function (data) {
+        $('#lava_select_dialog').html(data);
+    });
+}
 
=== added directory 'dashboard/lava'
=== added file 'dashboard/lava/__init__.py'
=== added directory 'dashboard/lava/forms'
=== added file 'dashboard/lava/forms/__init__.py'
=== added directory 'dashboard/lava/lib'
=== added file 'dashboard/lava/lib/__init__.py'
=== added file 'dashboard/lava/lib/lavalib.py'
--- dashboard/lava/lib/lavalib.py	1970-01-01 00:00:00 +0000
+++ dashboard/lava/lib/lavalib.py	2012-09-21 15:30:40 +0000
@@ -0,0 +1,249 @@
+# Copyright (C) 2012 Linaro
+#
+# This file is part of linaro-ci-dashboard.
+#
+# linaro-ci-dashboard is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# linaro-ci-dashboard is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with linaro-ci-dashboard.  If not, see <http://www.gnu.org/licenses/>.
+
+import xmlrpclib
+from lib.logger import Logger
+from dashboard.settings import LAVA_URL
+
+
+# The default delimiter used to separate test names to be executed.
+TEST_PLAN_DELIMITER = ';'
+
+# Map a TARGET_PRODUCT to LAVA parameters.
+PRODUCT_MAP = {
+    "pandaboard": {
+        "test_device_type": "panda",
+        "image_path": "%s%s" % (
+            "target/product/",
+            "pandaboard")},
+    "full_panda": {
+        "test_device_type": "panda",
+        "image_path": "%s%s" % (
+            "target/product/",
+            "panda")},
+    "beagleboard": {
+        "test_device_type": "beaglexm",
+        "image_path": "%s%s" % (
+            "target/product/",
+            "beagleboard")},
+    "snowball": {
+        "test_device_type": "snowball_sd",
+        "image_path": "%s%s" % (
+            "target/product/",
+            "snowball")},
+    "iMX53": {
+        "test_device_type": "mx53loco",
+        "image_path": "%s%s" % (
+            "target/product/",
+            "iMX53")},
+    "origen": {
+        "test_device_type": "origen",
+        "image_path": "%s%s" % (
+            "target/product/",
+            "origen")},
+    "vexpress": {
+        "test_device_type": "vexpress",
+        "image_path": "%s%s" % (
+            "target/product/",
+            "vexpress")},
+    }
+
+OPTION_SUFFIX = "_OPTION"
+TIMEOUT_SUFFIX = "_TIMEOUT"
+
+# Special token for LAVA_TEST_PLAN that allows the build owner to encode a
+# reboot in between test actions.
+REBOOT_TOKEN = '[system-reboot]'
+
+
+class LavaLibException(Exception):
+    """
+    Base class for exception in LAVA lib.
+    """
+
+
+class LavaLib(object):
+    """
+    Library to interact with LAVA XML-RPC.
+    """
+
+    def __init__(self, user=None, token=None, secure=True):
+        """
+        Initialize the LavaLib library to interact with LAVA.
+
+        :param user: The user name to use for interacting with LAVA.
+        :type user str
+        :param token: The token associated with the user.
+        :type token str
+        :param secure: If the connection should be secure or not. Default to
+        True.
+        :type secure bool
+        """
+        super(LavaLib, self).__init__()
+        self.log = Logger.getClassLogger(self)
+        self.user = user
+        self.token = token
+        self.secure = secure
+
+    def _lava_auth(self):
+        """
+        Creates the LAVA authorization string used to talk to LAVA.
+
+        :return A string with user name and token, if they have been specified.
+        """
+        auth_url = ''
+        if self.user is not None and self.token is not None:
+            auth_url = '%(user)s:%(token)s@' % dict(user=self.user,
+                                                    token=self.token)
+        return auth_url
+
+    def _create_server_url(self):
+        """
+        Creates the server URL necessary to interact with the LAVA server.
+
+        :return A string with the URL for the LAVA server.
+        """
+        protocol = 'https'
+        if not self.secure:
+            protocol = 'http'
+
+        lava_auth = self._lava_auth()
+
+        if lava_auth is not '' and not self.secure:
+            self.log.warning('Sending sensible information through an '
+                             'insecure channel. Consider using HTTPS.')
+
+        params_dict = dict(protocol=protocol,
+                           auth_url=lava_auth,
+                           lava_url=LAVA_URL)
+        server_url = ('%(protocol)s://%(auth_url)s%(lava_url)s' % params_dict)
+        return server_url
+
+    def _get_server(self):
+        """
+        Gets a connection to the LAVA server.
+
+        :return A xmlrpclib.ServerProxy object.
+        """
+        return xmlrpclib.ServerProxy(self._create_server_url())
+
+    def get_test_names(self, sort=True):
+        """
+        Get the name of the tests it is possible to run.
+
+        :param sort: If the results should be sorted alphabetically. Default to
+        True.
+        :type sort bool
+        :return A list with all the test names.
+        """
+        # TODO we are faking some data here, since the LAVA server at
+        # validation.linaro.org does not include now the function to get
+        # out the test names.
+        test_names = ['0xbench', 'bluetooth', 'bootchart', 'busybox',
+                      'glmark2', 'skia', 'v8', 'mmtest', 'monkey',
+                      'monkey_long_run']
+        if sort:
+            test_names.sort()
+        return test_names
+
+    def send_job(self, test_plan):
+        """
+        Sends the test plan to be executed to the LAVA server.
+
+        :param test_plan: The string of tests to be executed.
+        :type test_plan str
+        """
+        config = self._prepare_config(self._split_test_plan(test_plan))
+        try:
+            self._get_server().scheduler.submit_job(config)
+        except:
+            self.log.exception('Impossible to submit LAVA requests.')
+
+    def _split_test_plan(self, test_plan):
+        """
+        Splits the string of tests into single test names. The split happens
+        on the default delimiter set to the variable TEST_PLAN_DELIMITER.
+
+        :param test_plan: The string of tests to be split.
+        :type test_plan str
+        :return A list of test names.
+        """
+        if TEST_PLAN_DELIMITER in test_plan:
+            # Exclude the last element, the test plan we get should have also
+            # the delimiter appended on the last element.
+            tests = test_plan.split(TEST_PLAN_DELIMITER)[:-1]
+
+            # XXX need to check if this test still exists.
+            if "test_android_0xbench" in tests:
+                i = tests.index("test_android_0xbench")
+                tests[i] = "0xbench"
+        else:
+            self.log.exception('Found wrong delimiter in test plan '
+                               'definition.')
+            raise LavaLibException('Wrong delimiter in test plan string'
+                                   'definition.')
+        return tests
+
+    def _prepare_config(self, tests):
+        """
+        :return A JSON representation of the config to be run.
+        """
+        # TODO here should go the biggest part of the main() in
+        # post-build-lava inside linaro-android-build-tools.
+        return {}
+
+    def gen_android_test_actions(self, tests):
+        # TODO this needs to be reworked, need to eliminate env var lookup
+        # Method taken from linaro-android-build-tools.
+        import os
+        actions = []
+        if len(tests) > 0:
+            test_actions = [test for test in tests if test != REBOOT_TOKEN]
+
+            if len(test_actions) > 0:
+                inst_action = {
+                    "command": "lava_android_test_install",
+                    "parameters": {
+                        # ensure only unique test names
+                        "tests": list(set(test_actions))
+                    }
+                }
+                actions.append(inst_action)
+
+            for test in tests:
+                parameters = {'test_name': test}
+
+                test_option = os.environ.get('%s%s' % (test.upper(),
+                                                       OPTION_SUFFIX))
+                if test_option:
+                    parameters['option'] = test_option
+
+                timeout_option = os.environ.get('%s%s' % (test.upper(),
+                                                  TIMEOUT_SUFFIX))
+                if timeout_option:
+                    parameters['timeout'] = int(timeout_option)
+
+                if test != REBOOT_TOKEN:
+                    run_action = {
+                        "command": "lava_android_test_run",
+                        "parameters": parameters
+                    }
+                    actions.append(run_action)
+                else:
+                    actions.append({"command": "boot_linaro_android_image"})
+
+        return actions
 
=== added directory 'dashboard/lava/migrations'
=== added file 'dashboard/lava/migrations/0001_initial.py'
--- dashboard/lava/migrations/0001_initial.py	1970-01-01 00:00:00 +0000
+++ dashboard/lava/migrations/0001_initial.py	2012-09-21 15:30:40 +0000
@@ -0,0 +1,65 @@
+# encoding: utf-8
+import datetime
+from south.db import db
+from south.v2 import SchemaMigration
+from django.db import models
+
+class Migration(SchemaMigration):
+
+    def forwards(self, orm):
+        
+        # Adding model 'LavaTest'
+        db.create_table('lava_lavatest', (
+            ('id', self.gf('django.db.models.fields.AutoField')(primary_key=True)),
+            ('build', self.gf('django.db.models.fields.related.ForeignKey')(to=orm['frontend.LoopBuild'])),
+            ('test_name', self.gf('django.db.models.fields.CharField')(max_length=64)),
+            ('test_status', self.gf('django.db.models.fields.CharField')(max_length=20)),
+        ))
+        db.send_create_signal('lava', ['LavaTest'])
+
+
+    def backwards(self, orm):
+        
+        # Deleting model 'LavaTest'
+        db.delete_table('lava_lavatest')
+
+
+    models = {
+        'frontend.loop': {
+            'Meta': {'object_name': 'Loop'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'is_official': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'is_restricted': ('django.db.models.fields.BooleanField', [], {'default': 'False'}),
+            'name': ('django.db.models.fields.CharField', [], {'unique': 'True', 'max_length': '200'}),
+            'next_loop': ('django.db.models.fields.related.OneToOneField', [], {'blank': 'True', 'related_name': "'previous_loop'", 'unique': 'True', 'null': 'True', 'to': "orm['frontend.Loop']"}),
+            'server': ('django.db.models.fields.related.ForeignKey', [], {'default': 'None', 'to': "orm['jenkinsserver.JenkinsServer']", 'null': 'True'}),
+            'type': ('django.db.models.fields.CharField', [], {'max_length': '50', 'blank': 'True'})
+        },
+        'frontend.loopbuild': {
+            'Meta': {'ordering': "['-id']", 'unique_together': "(('build_number', 'loop'),)", 'object_name': 'LoopBuild'},
+            'build_number': ('django.db.models.fields.IntegerField', [], {'default': 'None'}),
+            'duration': ('django.db.models.fields.DecimalField', [], {'max_digits': '8', 'decimal_places': '2'}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'lava_test_status': ('django.db.models.fields.CharField', [], {'max_length': '20', 'null': 'True', 'blank': 'True'}),
+            'loop': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['frontend.Loop']"}),
+            'remote_number': ('django.db.models.fields.IntegerField', [], {'null': 'True', 'blank': 'True'}),
+            'result_xml': ('django.db.models.fields.TextField', [], {'null': 'True', 'blank': 'True'}),
+            'status': ('django.db.models.fields.CharField', [], {'max_length': '20'})
+        },
+        'jenkinsserver.jenkinsserver': {
+            'Meta': {'object_name': 'JenkinsServer'},
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'password': ('django.db.models.fields.CharField', [], {'max_length': '100'}),
+            'url': ('django.db.models.fields.CharField', [], {'max_length': '255'}),
+            'username': ('django.db.models.fields.CharField', [], {'max_length': '100'})
+        },
+        'lava.lavatest': {
+            'Meta': {'object_name': 'LavaTest'},
+            'build': ('django.db.models.fields.related.ForeignKey', [], {'to': "orm['frontend.LoopBuild']"}),
+            'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
+            'test_name': ('django.db.models.fields.CharField', [], {'max_length': '64'}),
+            'test_status': ('django.db.models.fields.CharField', [], {'max_length': '20'})
+        }
+    }
+
+    complete_apps = ['lava']
 
=== added file 'dashboard/lava/migrations/__init__.py'
=== added directory 'dashboard/lava/models'
=== added file 'dashboard/lava/models/__init__.py'
--- dashboard/lava/models/__init__.py	1970-01-01 00:00:00 +0000
+++ dashboard/lava/models/__init__.py	2012-09-21 15:30:40 +0000
@@ -0,0 +1,1 @@
+from dashboard.lava.models.lava import LavaTest
 
=== added file 'dashboard/lava/models/lava.py'
--- dashboard/lava/models/lava.py	1970-01-01 00:00:00 +0000
+++ dashboard/lava/models/lava.py	2012-09-21 15:30:40 +0000
@@ -0,0 +1,37 @@
+# Copyright (C) 2012 Linaro
+#
+# This file is part of linaro-ci-dashboard.
+#
+# linaro-ci-dashboard is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# linaro-ci-dashboard is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with linaro-ci-dashboard.  If not, see <http://www.gnu.org/licenses/>.
+
+from django.db import models
+from lib.logger import Logger
+from frontend.models.loop_build import LoopBuild
+
+
+class LavaTest(models.Model):
+    class Meta:
+        app_label = 'lava'
+
+    def __init__(self, *args, **kwargs):
+        self.log = Logger.getClassLogger(self)
+        super(LavaTest, self).__init__(*args, **kwargs)
+
+    PENDING = 'pending'
+    RUNNING = 'running'
+    FINISHED = 'finished'
+
+    build = models.ForeignKey(LoopBuild)
+    test_name = models.CharField(max_length=64)
+    test_status = models.CharField(max_length=20)
 
=== added directory 'dashboard/lava/templates'
=== added file 'dashboard/lava/templates/__init__.py'
=== added file 'dashboard/lava/templates/lava_select.html'
--- dashboard/lava/templates/lava_select.html	1970-01-01 00:00:00 +0000
+++ dashboard/lava/templates/lava_select.html	2012-09-21 15:30:40 +0000
@@ -0,0 +1,66 @@
+    <div id="tests_selection">
+        <div id="left_div">
+            <div id="loading_div" class="loading">
+                <img src="/img/loading.gif" alt="Loading test names..." />
+            </div>
+            <select id="available" name="name_available_select" multiple="multiple">
+                {% include "tests_list.html" %}
+            </select>
+        </div>
+        <div id="center_div">
+            <input type="button" id="add_button" value="&#8594;" />
+            <input type="button" id="remove_button" value="&#8592;" />
+        </div>
+        <div id="right_div">
+            <select id="chosen" multiple="multiple">
+                {% comment %}
+                This will be populated moving elements from the previous select
+                element..
+                {% endcomment %}
+            </select>
+        </div>
+    </div>
+    <div><input type="button" id="id_select_button" value="OK"/></div>
+<script type="text/javascript">
+    "use strict";
+    $.ajax({
+        url: "{% if request.is_secure %}https{% else %}http{% endif %}://{{ request.get_host }}{% url LavaSelectTestsNames %}",
+        beforeSend: function () {
+            // Do not show the select element if it is empty, just show the
+            // spinner.
+            $('#available').hide();
+        },
+        success: function (data) {
+            $(data).prependTo($('#available'));
+            $('#available').show();
+        },
+        error: function(data, status, error) {
+            $('#left_div').html('<span class="error"><strong>Oops!</strong> The LAVA server is not responding.</span>')
+        }
+    });
+    $('#loading_div').show().ajaxStop(function () {
+        // Disable the spinner once AJAX is done.
+        $(this).hide();
+    });
+    $('#add_button').click(function () {
+        return !$('#available option:selected').remove().appendTo('#chosen');
+    });
+    $('#remove_button').click(function () {
+        return !$('#chosen option:selected').remove().appendTo('#available');
+    });
+    $('#id_select_button').click(function () {
+        var values = "";
+        $("#chosen > option").each(function() {
+            // We store the values as a semi-colon separated list of names.
+            values += $(this).val() + ";";
+        });
+        $('#id_lava_tests_0').val(values);
+        // Ellipsize the visual representation if it exceeds a prefixed amount
+        // of chars in length, and append an ellipsis.
+        if (values.length > 30) {
+            values = values.substring(0, 27) + "&#8230;";
+        }
+        $('#lava_select_link').html(values);
+        $('#lava_select_dialog').dialog('close');
+    });
+</script>
 
=== added file 'dashboard/lava/templates/tests_list.html'
--- dashboard/lava/templates/tests_list.html	1970-01-01 00:00:00 +0000
+++ dashboard/lava/templates/tests_list.html	2012-09-21 15:30:40 +0000
@@ -0,0 +1,7 @@
+{% comment %}
+This is a basic template used to populate a select element in another view
+through an asynchronous call.
+{% endcomment %}
+{% for test in tests %}
+    <option id="id_{{ test }}" value="{{ test }}">{{ test }}</option>
+{% endfor %}
 \ No newline at end of file
 
=== added directory 'dashboard/lava/tests'
=== added file 'dashboard/lava/tests/__init__.py'
--- dashboard/lava/tests/__init__.py	1970-01-01 00:00:00 +0000
+++ dashboard/lava/tests/__init__.py	2012-09-21 15:30:40 +0000
@@ -0,0 +1,23 @@
+# Copyright (C) 2012 Linaro
+#
+# This file is part of linaro-ci-dashboard.
+#
+# linaro-ci-dashboard is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# linaro-ci-dashboard is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with linaro-ci-dashboard.  If not, see <http://www.gnu.org/licenses/>.
+
+from dashboard.lava.tests.test_lavalib import *
+
+
+__test__ = {
+    'LavalibTest': LavalibTest,
+    }
 
=== added file 'dashboard/lava/tests/test_lavalib.py'
--- dashboard/lava/tests/test_lavalib.py	1970-01-01 00:00:00 +0000
+++ dashboard/lava/tests/test_lavalib.py	2012-09-21 15:30:40 +0000
@@ -0,0 +1,70 @@
+# Copyright (C) 2012 Linaro
+#
+# This file is part of linaro-ci-dashboard.
+#
+# linaro-ci-dashboard is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# linaro-ci-dashboard is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with linaro-ci-dashboard.  If not, see <http://www.gnu.org/licenses/>.
+
+from django.test import TestCase
+from dashboard.lava.lib.lavalib import (
+    LavaLib,
+    LavaLibException,
+    LAVA_URL,
+)
+
+
+class LavalibTest(TestCase):
+
+    def setUp(self):
+        self.user = 'auser'
+        self.token = 'atoken'
+        self.lava_lib = LavaLib(self.user, self.token)
+
+    def test_split_test_plan(self):
+        test_plan = 'a;b;c;d;'
+        expected_out = ['a', 'b', 'c', 'd']
+        self.assertEqual(expected_out,
+                         self.lava_lib._split_test_plan(test_plan))
+
+    def test_split_test_plan_wrong(self):
+        test_plan = 'a,b,c'
+        self.assertRaises(LavaLibException,
+                          self.lava_lib._split_test_plan, test_plan)
+
+    def test_create_lava_auth(self):
+        expected_out = ('%s:%s@' % (self.user, self.token))
+        self.assertEqual(expected_out, self.lava_lib._lava_auth())
+
+    def test_create_lava_auth_empty(self):
+        lava_lib = LavaLib()
+        expected_out = ''
+        self.assertEqual(expected_out, lava_lib._lava_auth())
+
+    def test_create_server_url(self):
+        expected_out = ('https://%s:%s@' + LAVA_URL) % (self.user, self.token)
+        self.assertEqual(expected_out, self.lava_lib._create_server_url())
+
+    def test_create_server_url_no_user(self):
+        expected_out = 'https://' + LAVA_URL
+        lava_lib = LavaLib()
+        self.assertEqual(expected_out, lava_lib._create_server_url())
+
+    def test_create_server_url_not_secure_no_user(self):
+        expected_out = 'http://' + LAVA_URL
+        lava_lib = LavaLib(secure=False)
+        self.assertEqual(expected_out, lava_lib._create_server_url())
+
+    def test_create_server_url_not_secure_with_user(self):
+        expected_out = ('http://%s:%s@' + LAVA_URL) % (self.user, self.token)
+        lava_lib = LavaLib(user=self.user, token=self.token, secure=False)
+        self.assertEqual(expected_out, lava_lib._create_server_url())
 
=== added file 'dashboard/lava/urls.py'
--- dashboard/lava/urls.py	1970-01-01 00:00:00 +0000
+++ dashboard/lava/urls.py	2012-09-21 15:30:40 +0000
@@ -0,0 +1,29 @@
+#!/usr/bin/env python
+# Copyright (C) 2012 Linaro
+#
+# This file is part of linaro-ci-dashboard.
+#
+# linaro-ci-dashboard is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# linaro-ci-dashboard is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with linaro-ci-dashboard.  If not, see <http://www.gnu.org/licenses/>.
+
+from django.conf.urls.defaults import patterns, url
+from dashboard.lava.views.lava_select_view import LavaSelectView
+from dashboard.lava.views.lava_select_test_names import LavaSelectTestNames
+
+
+urlpatterns = patterns('',
+    url(r'^lava/select/$',
+        LavaSelectView.as_view(), name='LavaSelectView'),
+    url(r'^lava/ajax/get-test-names/$',
+        LavaSelectTestNames.as_view(), name='LavaSelectTestsNames'),
+    )
 \ No newline at end of file
 
=== added directory 'dashboard/lava/views'
=== added file 'dashboard/lava/views/__init__.py'
=== added file 'dashboard/lava/views/lava_select_test_names.py'
--- dashboard/lava/views/lava_select_test_names.py	1970-01-01 00:00:00 +0000
+++ dashboard/lava/views/lava_select_test_names.py	2012-09-21 15:30:40 +0000
@@ -0,0 +1,58 @@
+# Copyright (C) 2012 Linaro
+#
+# This file is part of linaro-ci-dashboard.
+#
+# linaro-ci-dashboard is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# linaro-ci-dashboard is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with linaro-ci-dashboard.  If not, see <http://www.gnu.org/licenses/>.
+
+from django.template import RequestContext
+from django.shortcuts import render_to_response
+from django.contrib.auth.decorators import login_required
+from django.views.generic.base import TemplateView
+from django.utils.decorators import method_decorator
+from lava.lib.lavalib import LavaLib
+
+
+class LavaSelectTestNames(TemplateView):
+    """
+    View class used to make asynchronous calls in order to retrieve the list of
+    available test names from a LAVA server through the LAVA library.
+
+    This is not a real view, it is used only to make async call into the
+    ci-dashboard server.
+    """
+    template_name = "tests_list.html"
+
+    @method_decorator(login_required)
+    def dispatch(self, *args, **kwargs):
+        return super(LavaSelectTestNames, self).dispatch(*args, **kwargs)
+
+    def get_lava_tests(self):
+        """
+        Retrieves all the test names from the LAVA server.
+
+        :return A list with all the test names.
+        """
+        lava_server = LavaLib(secure=False)
+        return lava_server.get_test_names()
+
+    def render_to_response(self, context, **response_kwargs):
+        if self.request.is_ajax():
+            data = {
+                "tests": self.get_lava_tests()
+            }
+            return render_to_response(self.template_name, data,
+                                      context_instance=RequestContext(
+                                          self.request))
+        else:
+            raise NotImplementedError
 
=== added file 'dashboard/lava/views/lava_select_view.py'
--- dashboard/lava/views/lava_select_view.py	1970-01-01 00:00:00 +0000
+++ dashboard/lava/views/lava_select_view.py	2012-09-21 15:30:40 +0000
@@ -0,0 +1,46 @@
+# Copyright (C) 2012 Linaro
+#
+# This file is part of linaro-ci-dashboard.
+#
+# linaro-ci-dashboard is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# linaro-ci-dashboard is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with linaro-ci-dashboard.  If not, see <http://www.gnu.org/licenses/>.
+
+from django.contrib.auth.decorators import login_required
+from django.views.generic.base import TemplateView
+from django.utils.decorators import method_decorator
+from django.shortcuts import render_to_response
+from django.template import RequestContext
+
+
+class LavaSelectView(TemplateView):
+    template_name = 'lava_select.html'
+
+    @method_decorator(login_required)
+    def dispatch(self, *args, **kwargs):
+        return super(LavaSelectView, self).dispatch(*args, **kwargs)
+
+    def get_context_data(self, **kwargs):
+        context = super(LavaSelectView, self).get_context_data(**kwargs)
+        context['request'] = self.request
+        return context
+
+    def render_to_response(self, context, **response_kwargs):
+        data = {
+            "request": self.request
+        }
+        if self.request.is_ajax():
+            return render_to_response(self.template_name, data,
+                                      context_instance=RequestContext(
+                                          self.request))
+        else:
+            raise NotImplementedError
 
=== added directory 'dashboard/lava/widgets'
=== added file 'dashboard/lava/widgets/__init__.py'
=== added file 'dashboard/lava/widgets/lava_test_select_widget.py'
--- dashboard/lava/widgets/lava_test_select_widget.py	1970-01-01 00:00:00 +0000
+++ dashboard/lava/widgets/lava_test_select_widget.py	2012-09-21 15:30:40 +0000
@@ -0,0 +1,54 @@
+# Copyright (C) 2012 Linaro
+#
+# This file is part of linaro-ci-dashboard.
+#
+# linaro-ci-dashboard is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# linaro-ci-dashboard is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with linaro-ci-dashboard.  If not, see <http://www.gnu.org/licenses/>.
+
+from django.forms.widgets import (
+    HiddenInput,
+    MultiWidget,
+    Widget
+)
+
+
+class LavaLinkWidget(Widget):
+    def render(self, name, value, attrs=None):
+        from django.utils.safestring import mark_safe
+        if not value:
+            value = "Select Tests"
+        output = ['<span><a href="#" id="lava_select_link">%s</a>'
+                  '</span>' % value]
+        return mark_safe(u' '.join(output))
+
+
+class LavaTestSelectWidget(MultiWidget):
+    class Media:
+        js = ("/js/lava/lava_test_select.js",)
+
+    def __init__(self, attrs=None):
+        _widgets = (
+            HiddenInput(attrs=attrs),
+            LavaLinkWidget(attrs=attrs),
+            )
+        super(LavaTestSelectWidget, self).__init__(_widgets, attrs)
+
+    def decompress(self, value):
+        rtn_list = [None, None]
+        return rtn_list
+
+    def format_output(self, rendered_widgets):
+        return u''.join(rendered_widgets)
+
+    def value_from_datadict(self, data, files, name):
+        return data.get("lava_tests_0", None)
 
=== modified file 'dashboard/settings.py'
--- dashboard/settings.py	2012-09-21 09:25:19 +0000
+++ dashboard/settings.py	2012-09-21 15:30:40 +0000
@@ -29,6 +29,7 @@
 DEPS_INSTALL_PATH = os.path.join(ROOT_PATH, "..", "lib")
 JS_PATH = os.path.join(ROOT_PATH, "js")
 CSS_PATH = os.path.join(ROOT_PATH, "css")
+IMG_PATH = os.path.join(ROOT_PATH, 'img')
 
 ADMINS = (
     # ('Your Name', '[e-mail]'),
@@ -142,6 +143,7 @@
     'django.contrib.messages',
     'django.contrib.staticfiles',
     'django_openid_auth',
+    'lava',
     'jenkinsserver',
     'frontend',
     'frontend.android_build',
@@ -224,6 +226,12 @@
 JENKINS_HOME = os.path.join(os.getenv('HOME'), ".jenkins_test")
 JENKINS_LOG_NAME = 'jenkins.log'
 JENKINS_DB_ID = 1
+<<<<<<< TREE
 
 LAVA_ACCEPTING_ADDRESS = '0.0.0.0'
 LAVA_PORT = '8001'
+=======
+
+# LAVA specific values
+LAVA_URL = 'validation.linaro.org/lava-server/RPC2/'
+>>>>>>> MERGE-SOURCE
 
=== modified file 'dashboard/urls.py'
--- dashboard/urls.py	2012-09-13 10:51:03 +0000
+++ dashboard/urls.py	2012-09-21 15:30:40 +0000
@@ -28,9 +28,12 @@
     url(r'^admin/', include(admin.site.urls)),
     url(r'^logout/$', 'django.contrib.auth.views.logout'),
     url(r'^js/(?P<path>.*)$', 'django.views.static.serve',
-     {'document_root': settings.JS_PATH}),
+            {'document_root': settings.JS_PATH}),
     url(r'^css/(?P<path>.*)$', 'django.views.static.serve',
-     {'document_root': settings.CSS_PATH}),
+            {'document_root': settings.CSS_PATH}),
+    url(r'^img/(?P<path>.*)$', 'django.views.static.serve',
+            {'document_root': settings.IMG_PATH}),
+    url(r'^', include('dashboard.lava.urls')),
     url(r'^', include('dashboard.frontend.urls')),
     url(r'^', include('dashboard.frontend.android_build.urls')),
     url(r'^', include('dashboard.frontend.kernel_build.urls')),