Merge lp:~doanac/qa-dashboard/live-status into lp:qa-dashboard

[Andy Doan]

Add a REST API to Smoke objects

Add a REST API to Smoke objects

This gives us the ability to have our jenkins jobs notify the
dashboard in realtime about smoke status and not have to wait
until the job has been pushed to the public server and then
polled by the pull-script.

In addition to a fairly thorough set of unit-tests, I've also
created a pretty easy CLI to do some exploratory testing with:

 http://paste.ubuntu.com/6454113/

That script will probably land in the lp:ubuntu-test-cases/touch repo
in some form or another.

Some notes to keep in mind:

* The API does not allow for delete operations (just to help decrease
  the amount of damage someone might accidentally do).

* It doesn't require auth[entication|orization] for GETS, but
  use's Django users + Tastypie ApiKey for PUT and PATCH operations.

* I had to use two hacks with Tastypie:
  1) 0.9.15 because 0.10 migrations don't work with python 2.7 and
     django 1.5

  2) settings.US_TZ had to be enabled as noted in the settings.py
     due to a tastypie bug.

I'm not really excited about #2, but given the limited use of "writes"
we have in the dashboard, I think this may be fairly safe/isolated.

[Andy Doan (doanac)]

wanted to share the branch, but I'd like to try and get one more unit test written to help make sure it plays nicely with the pull-script

[Andy Doan (doanac)]

we are blocked on:

 https://rt.admin.canonical.com/Ticket/Display.html?id=66058

[Andy Doan (doanac)]

tastypie 0.9.15 should be ready, so this can be reviewed now.

[Andy Doan (doanac)]

here's a branch to give you guys some insight into how i use this API:

 http://bazaar.launchpad.net/~doanac/ubuntu-test-cases/live-status/revision/128

this is all working at home now. The remaining work I need to look at is getting our smoke pull script to handle multiple results from a single job.

[Paul Larson (pwlars)]

> here's a branch to give you guys some insight into how i use this API:
>
> http://bazaar.launchpad.net/~doanac/ubuntu-test-cases/live-
> status/revision/128
>
> this is all working at home now. The remaining work I need to look at is
> getting our smoke pull script to handle multiple results from a single job.
Really cool! Should we have to care about the pull script though? It should just replace the current result with what it pulls. So if it was in progress before, or even complete before, it's only going to get the "DONE" status for a fully complete job on a pull. At worst, it should be the same as what it has.

Or do you mean that pulling a job that was in a still running state before would be marked as still running?

[Andy Doan (doanac)]

On 12/06/2013 11:51 AM, Paul Larson wrote:
> Really cool! Should we have to care about the pull script though? It should just replace the current result with what it pulls. So if it was in progress before, or even complete before, it's only going to get the "DONE" status for a fully complete job on a pull. At worst, it should be the same as what it has.
>
> Or do you mean that pulling a job that was in a still running state before would be marked as still running?

The pulls script also learns about all the job artifacts. This is the
key thing we need before marking a result as "COMPLETE".

[Joe Talbott (joetalbott)]

Only thing I see is the 2012 in the copyright.

Otherwise +1.

review: Approve

[Joe Talbott (joetalbott)]

review: Approve

[Chris Johnston (cjohnston)]

review: Approve

[Chris Johnston (cjohnston)]

The attempt to merge lp:~doanac/qa-dashboard/live-status into lp:qa-dashboard failed. Below is the output from the failed tests.

Traceback (most recent call last):
  File "./manage.py", line 11, in <module>
    execute_from_command_line(sys.argv)
  File "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line 453, in execute_from_command_line
    utility.execute()
  File "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line 392, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line 272, in fetch_command
    klass = load_command_class(app_name, subcommand)
  File "/usr/lib/python2.7/dist-packages/django/core/management/__init__.py", line 77, in load_command_class
    module = import_module('%s.management.commands.%s' % (app_name, name))
  File "/usr/lib/python2.7/dist-packages/django/utils/importlib.py", line 35, in import_module
    __import__(name)
  File "/usr/lib/python2.7/dist-packages/south/management/commands/__init__.py", line 10, in <module>
    import django.template.loaders.app_directories
  File "/usr/lib/python2.7/dist-packages/django/template/loaders/app_directories.py", line 25, in <module>
    raise ImproperlyConfigured('ImportError %s: %s' % (app, e.args[0]))
django.core.exceptions.ImproperlyConfigured: ImportError tastypie: No module named tastypie

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:692
http://s-jenkins.ubuntu-ci:8080/job/dashboard-ci/274/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/dashboard-ci/274/rebuild

review: Needs Fixing (continuous-integration)

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:692
http://s-jenkins.ubuntu-ci:8080/job/dashboard-ci/275/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/dashboard-ci/275/rebuild

review: Needs Fixing (continuous-integration)

[lp:~doanac/qa-dashboard/live-status]

693. By Andy Doan on 2013-12-12

a hack to deal with tastypie bug

setting USE_TZ=True is pain. It causes lots of django warnings and
broke some of our exsiting test case logic. This backs out the
settings change for USE_TZ and monkey-patches the actual buggy spot
in code so that we can use the admin panel properly

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:693
http://s-jenkins.ubuntu-ci:8080/job/dashboard-ci/276/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/dashboard-ci/276/rebuild

review: Needs Fixing (continuous-integration)

[lp:~doanac/qa-dashboard/live-status]

694. By Andy Doan on 2013-12-12

merge with trunk

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:694
http://s-jenkins.ubuntu-ci:8080/job/dashboard-ci/277/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/dashboard-ci/277/rebuild

review: Approve (continuous-integration)

[Joe Talbott (joetalbott)]

review: Approve

=== modified file 'qa_dashboard/settings.py'
--- qa_dashboard/settings.py	2013-12-12 04:39:02 +0000
+++ qa_dashboard/settings.py	2013-12-12 05:37:33 +0000
@@ -192,6 +192,7 @@
     'django.contrib.staticfiles',
     'django_tables2',
     'south',
+    'tastypie',   # needed for ApiKeyAuthentication in the admin panel
 )
 
 TEST_RUNNER = "qa_dashboard.local_tests.LocalAppsTestSuiteRunner"
@@ -323,3 +324,15 @@
     pass
 
 INSTALLED_APPS = PLUGIN_APPS + LOCAL_APPS + INSTALLED_APPS
+
+
+# for tastypie, a hack to allow us to not require using TZ:
+# http://github.com/toastdriven/django-tastypie/pull/561#issuecomment-26204496
+def _now():
+    from django.utils import timezone
+    d = timezone.now()
+    if d.tzinfo:
+        d = timezone.localtime(timezone.now())
+    return d
+import tastypie.models
+tastypie.models.ApiKey._meta.get_field_by_name('created')[0].default = _now

=== modified file 'requirements.txt'
--- requirements.txt	2013-05-20 16:36:00 +0000
+++ requirements.txt	2013-12-12 05:37:33 +0000
@@ -11,3 +11,6 @@
 django-openid-auth==0.5
 model-mommy==1.0
 mock==1.0.1
+# NOTE: django-tastypie 0.10 has a bug that prevents south migrations
+# from working for us.
+django-tastypie==0.9.15

=== added file 'smokeng/api.py'
--- smokeng/api.py	1970-01-01 00:00:00 +0000
+++ smokeng/api.py	2013-12-12 05:37:33 +0000
@@ -0,0 +1,119 @@
+# QA Dashboard
+# Copyright 2012-2013 Canonical Ltd.
+
+# This program is free software: you can redistribute it and/or modify it
+# under the terms of the GNU Affero General Public License version 3, as
+# published by the Free Software Foundation.
+
+# This program is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranties of
+# MERCHANTABILITY, SATISFACTORY QUALITY, or FITNESS FOR A PARTICULAR
+# PURPOSE.  See the GNU Affero General Public License for more details.
+
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+
+from tastypie import fields
+from tastypie.api import Api
+from tastypie.authentication import ApiKeyAuthentication
+from tastypie.authorization import DjangoAuthorization
+from tastypie.resources import ModelResource, ALL, ALL_WITH_RELATIONS
+
+from common.models import JenkinsBuild, JenkinsJob
+from smokeng.models import SmokeImage, SmokeResult
+
+
+# don't allow deletion of objects with this API
+ALLOWED_METHODS = ['get', 'post', 'patch']
+
+
+class WriteAuthentication(ApiKeyAuthentication):
+    '''only require authentication for non-GET operations.'''
+    def is_authenticated(self, req, **kwargs):
+        if req.method == 'GET':
+            return True
+        return super(WriteAuthentication, self).is_authenticated(req, **kwargs)
+
+
+class WriteAuthorization(DjangoAuthorization):
+    '''only require authorization for non-GET operations.'''
+
+    def is_authorized(self, req, object=None):
+        if req.method == 'GET':
+            return True
+        return super(WriteAuthorization, self).is_authorized(req, object)
+
+
+class JenkinsJobResource(ModelResource):
+    class Meta:
+        queryset = JenkinsJob.objects.filter(publish=True)
+        resource_name = 'job'
+
+        filtering = {
+            'name': ALL,
+        }
+
+        authentication = WriteAuthentication()
+        authorization = WriteAuthorization()
+        allowed_methods = ALLOWED_METHODS
+
+
+class JenkinsBuildResource(ModelResource):
+    job = fields.ForeignKey(JenkinsJobResource, 'job')
+
+    class Meta:
+        queryset = JenkinsBuild.objects.filter(publish=True)
+        resource_name = 'build'
+
+        filtering = {
+            'job': ALL_WITH_RELATIONS,
+            'build_number': ALL,
+        }
+
+        authentication = WriteAuthentication()
+        authorization = WriteAuthorization()
+        allowed_methods = ALLOWED_METHODS
+
+
+class SmokeImageResource(ModelResource):
+    class Meta:
+        queryset = SmokeImage.objects.filter(publish=True)
+        resource_name = 'image'
+
+        filtering = {
+            'build_number': ALL,
+            'release': ALL,
+            'flavor': ALL,
+            'variant': ALL,
+            'arch': ALL,
+        }
+
+        authentication = WriteAuthentication()
+        authorization = WriteAuthorization()
+        allowed_methods = ALLOWED_METHODS
+
+
+class SmokeResultResource(ModelResource):
+    image = fields.ForeignKey(SmokeImageResource, 'image')
+    jenkins_build = fields.ForeignKey(JenkinsBuildResource, 'jenkins_build')
+
+    class Meta:
+        queryset = SmokeResult.objects.filter(publish=True)
+        resource_name = 'result'
+
+        filtering = {
+            'jenkins_build': ALL_WITH_RELATIONS,
+            'image': ALL_WITH_RELATIONS,
+            'name': ALL,
+        }
+
+        authentication = WriteAuthentication()
+        authorization = WriteAuthorization()
+        allowed_methods = ALLOWED_METHODS
+
+
+v1_api = Api(api_name='v1')
+v1_api.register(JenkinsJobResource())
+v1_api.register(JenkinsBuildResource())
+v1_api.register(SmokeImageResource())
+v1_api.register(SmokeResultResource())

=== modified file 'smokeng/tests.py'
--- smokeng/tests.py	2013-12-05 19:22:35 +0000
+++ smokeng/tests.py	2013-12-12 05:37:33 +0000
@@ -14,6 +14,7 @@
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
 import datetime
+import json
 import mock
 import urllib2
 
@@ -22,6 +23,9 @@
 from django.test.client import Client
 from django.contrib.auth.models import User
 
+from tastypie.models import ApiKey
+from tastypie.test import TestApiClient
+
 from smokeng.models import (
     SmokeImage,
     SmokeResult,
@@ -835,3 +839,187 @@
         resp = self.client.get('/smokeng/')
         pass_rate = resp.context[-1]['table'].rows[0]['pass_rate']
         self.assertEqual(u'Running', pass_rate.strip())
+
+
+class TestSmokeApi(TestCase):
+
+    def setUp(self):
+        _setUp(self)
+
+        # create an api key for "post" operations
+        u = User()
+        u.username = 'admin'
+        u.is_superuser = True
+        u.save()
+
+        k = ApiKey()
+        k.user = u
+        k.key = 'key'
+        k.save()
+        self.user = u.username
+        self.key = k.key
+
+        # make a basic assumption on what's already in the DB for
+        # tests below
+        self.assertEqual(1, SmokeResult.objects.all().count())
+        self.client = TestApiClient()
+
+    def _get(self, resource, params=None):
+        return json.loads(self.client.get(resource, data=params).content)
+
+    def _post(self, resource, params):
+        auth = 'ApiKey %s:%s' % (self.user, self.key)
+        resp = self.client.post(resource, data=params, authentication=auth)
+        self.assertEqual(201, resp.status_code)
+        return resp['location']
+
+    def _patch(self, resource, params):
+        auth = 'ApiKey %s:%s' % (self.user, self.key)
+        resp = self.client.patch(resource, data=params, authentication=auth)
+        self.assertEqual(202, resp.status_code)
+
+    def testBuildGet(self):
+        params = {
+            'build_number': self.jenkins_build.build_number,
+            'job': self.jenkins_job.id,
+        }
+        obj = self._get('/smokeng/api/v1/build/', params)
+        self.assertEqual(1, len(obj['objects']))
+        obj = obj['objects'][0]
+        self.assertEqual(self.jenkins_build.ran_at.isoformat(), obj['ran_at'])
+
+    def testImageGet(self):
+        params = {
+            'build_number': self.image.build_number,
+            'release': self.image.release,
+            'variant': self.image.variant,
+            'arch': self.image.arch,
+        }
+        obj = self._get('/smokeng/api/v1/image/', params)
+        self.assertEqual(1, len(obj['objects']))
+        obj = obj['objects'][0]
+        self.assertEqual(self.image.build_number, obj['build_number'])
+        self.assertEqual(self.image.release, obj['release'])
+
+    def testResultGet(self):
+        params = {
+            'jenkins_build': self.jenkins_build.id,
+            'image': self.image.id,
+            'name': self.result.name,
+        }
+        obj = self._get('/smokeng/api/v1/result/', params)
+        self.assertEqual(1, len(obj['objects']))
+        obj = obj['objects'][0]
+        self.assertEqual(self.result.name, obj['name'])
+        self.assertEqual(self.result.total_count, obj['total_count'])
+        self.assertEqual(self.result.pass_count, obj['pass_count'])
+        self.assertEqual(self.result.status, obj['status'])
+
+    def testBuildAdd(self):
+        params = {
+            'build_number': 'build_number',
+            'job': '/smokeng/api/v1/job/%d/' % self.jenkins_job.id,
+            'ran_at': datetime.datetime.now().isoformat(),
+            'build_description': 'inprogress',
+        }
+        loc = self._post('/smokeng/api/v1/build/', params)
+        obj = self._get(loc)
+        self.assertEqual(obj['build_number'], params['build_number'])
+        self.assertEqual(obj['build_description'], params['build_description'])
+        self.assertEqual(obj['job'], params['job'])
+        self.assertEqual(obj['ran_at'], params['ran_at'])
+
+    def testImageAdd(self):
+        params = {
+            'build_number': 'build_number',
+            'release': 'release',
+            'flavor': 'flavor',
+            'variant': 'variant',
+            'arch': 'arch',
+        }
+        loc = self._post('/smokeng/api/v1/image/', params)
+        obj = self._get(loc)
+        self.assertEqual(params['arch'], obj['arch'])
+        self.assertEqual(params['release'], obj['release'])
+        self.assertEqual(params['build_number'], obj['rootfs_id'])
+
+    def testResultAdd(self):
+        params = {
+            'ran_at': datetime.datetime.now().isoformat(),
+            'status': 0,
+            'total_count': 6,
+            'pass_count': 3,
+            'error_count': 2,
+            'fail_count': 1,
+            'jenkins_build': ('/smokeng/api/v1/build/%d/' %
+                              self.jenkins_build.id),
+            'image': '/smokeng/api/v1/image/%d/' % self.image.id,
+            'name': 'the-test-result-name',
+        }
+        loc = self._post('/smokeng/api/v1/result/', params)
+        obj = self._get(loc)
+        self.assertEqual(params['name'], obj['name'])
+        self.assertEqual(params['ran_at'], obj['ran_at'])
+        self.assertEqual(params['status'], obj['status'])
+        self.assertEqual(params['total_count'], obj['total_count'])
+        self.assertEqual(params['pass_count'], obj['pass_count'])
+        self.assertEqual(params['error_count'], obj['error_count'])
+        self.assertEqual(params['fail_count'], obj['fail_count'])
+        self.assertEqual(params['jenkins_build'], obj['jenkins_build'])
+        self.assertEqual(params['image'], obj['image'])
+
+    def _testResultState(self, state):
+        resource = '/smokeng/api/v1/result/%d/' % self.result.id
+        params = {
+            'ran_at': datetime.datetime.now().isoformat(),
+            'status': state,
+            'total_count': 6,
+            'pass_count': 3,
+            'error_count': 2,
+            'fail_count': 1,
+        }
+        self._patch(resource, params)
+        res = SmokeResult.objects.get(pk=self.result.id)
+        self.assertEqual(self.result.name, res.name)
+        self.assertEqual(state, res.status)
+
+    def testResultStates(self):
+        self._testResultState(SmokeResult.QUEUED)
+        self._testResultState(SmokeResult.RUNNING)
+        self._testResultState(SmokeResult.SYNCING)
+        self._testResultState(SmokeResult.COMPLETE)
+
+    def _testPullInteraction(self, as_string):
+        '''The ran_at field can be both strings and datetime objects'''
+        ran_at = datetime.datetime.now()
+        if as_string:
+            ran_at = ran_at.strftime('%Y-%m-%d %H:%M:%S')
+
+        data = {
+            # total/passes/errors/failures come from _testResultState
+            'test_name': self.result.name,
+            'total': 6,
+            'passes': 3,
+            'errors': 2,
+            'failures': 1,
+            'ran_at': ran_at,
+        }
+        self._testResultState(SmokeResult.SYNCING)
+        c = Command()
+        c.image = self.image
+        c.bugs = []
+        c.install_data = {
+            'test_name': self.result.name,
+            'jenkins_build': self.jenkins_build
+        }
+        c.add_result(data)
+
+        res = SmokeResult.objects.get(pk=self.result.id)
+        self.assertEqual(self.result.name, res.name)
+        self.assertEqual(SmokeResult.COMPLETE, res.status)
+
+    def testPullAsString(self):
+        self._testPullInteraction(True)
+
+    def testPullAsDatetime(self):
+        self._testPullInteraction(False)

=== modified file 'smokeng/urls.py'
--- smokeng/urls.py	2013-09-27 18:27:12 +0000
+++ smokeng/urls.py	2013-12-12 05:37:33 +0000
@@ -13,10 +13,14 @@
 # You should have received a copy of the GNU Affero General Public License
 # along with this program.  If not, see <http://www.gnu.org/licenses/>.
 
-from django.conf.urls import patterns, url
+from django.conf.urls import include, patterns, url
+
+from smokeng.api import v1_api
+
 
 urlpatterns = patterns(
     'smokeng.views',
+    url(r'^api/', include(v1_api.urls)),
     url(
         r'^(?:(?P<release>\w+)/)?(?:(?P<variant>\w+)/)?$',
         'overview',