Merge lp:~robru/cupstream2distro/add-ticket-system-support into lp:cupstream2distro

[Robert Bruce Park]

Teach CI Train how to push statuses to the ticket system.

This can be tested locally in an interactive python session by pasting the following lines one by one into a terminal after cd'ing into the branch dir:

    BUILD_URL=http://example.com DEBUG=true python
    from cupstream2distro.silomanager import SiloState
    s = SiloState.new_blank('ubuntu/some-ppa', requestid='e107b2fa-8625-11e4-98f1-fa163eb82757')
    s.set_ticket_status('Silo building', 'In progress')
    s.set_ticket_status('0', '0')

The output is expected to look something like this:

http://paste.ubuntu.com/9564610/

(the env spew at the import step is expected, don't worry about that)

And you can see the results reflected at:

http://172.19.1.177/ticket/e107b2fa-8625-11e4-98f1-fa163eb82757/

[Robert Bruce Park (robru)]

We don't seem to have access to bootstack from production:

https://ci-train.ubuntu.com/job/cyphermox-test/511/console

[Robert Bruce Park (robru)]

Hey guys, so most of this is pretty uncontroversial, however the ticket system seems to be missing a good analog for 'Silo merging', so I substituted 'Completed' instead, which is probably wrong but I'm not sure what else to do there.

Also, during the next phase when the ticket system grows to absorb the dashboard, there'll need to be a way to inject arbitrary status messages for the ticket system to display. As you can see in this merge there are many failure messages that get saved into the dashboard but not pushed into the ticket system, which is fine for now, but will need to get fixed later on.

[lp:~robru/cupstream2distro/add-ticket-system-support]

865. By Robert Bruce Park on 2014-12-18

Better mocks.

[Robert Bruce Park (robru)]

One thing we should probably discuss is the potential for redundancy here since there isn't a clear distinction between our steps and our statuses. So we've got two kinds of methods here, some go 'self.set_config_status(...); self.set_ticket_step(...)' and some go 'self.config_step = ...; self.set_ticket_step(...)'

I suspect that those two are always called in pairs, I'll have to check the jobs for any cases where a config_step might get called without a set_config_status equivalent at the same time. If that's the case then I'm really making two PATCH requests every time something changes, so I should probably check if those can be removed.

[lp:~robru/cupstream2distro/add-ticket-system-support]

866. By Robert Bruce Park on 2014-12-18

Tweak variable names.

[Robert Bruce Park (robru)]

Ok yeah, I audited the citrain/ scripts and the steps are totally redundant to the statuses. I'll drop those in order to reduce redundant calls into the ticket system.

[lp:~robru/cupstream2distro/add-ticket-system-support]

867. By Robert Bruce Park on 2014-12-18

Drop redundant calls to set_ticket_step().

868. By Robert Bruce Park on 2014-12-18

Rename set_ticket_step to set_ticket_status for consistency.

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:
http://jenkins.qa.ubuntu.com/job/cupstream2distro-config-ci/773/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/cupstream2distro-config-vivid-amd64-ci/8/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/cupstream2distro-config-vivid-i386-ci/8/console

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/cupstream2distro-config-ci/773/rebuild

review: Needs Fixing (continuous-integration)

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:
http://jenkins.qa.ubuntu.com/job/cu2d-choo-choo-ci/392/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/cu2d-choo-choo-ci/392/rebuild

review: Needs Fixing (continuous-integration)

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:868
http://jenkins.qa.ubuntu.com/job/cu2d-choo-choo-ci/393/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/cu2d-choo-choo-ci/393/rebuild

review: Needs Fixing (continuous-integration)

[PS Jenkins bot (ps-jenkins)]

FAILED: Continuous integration, rev:868
http://jenkins.qa.ubuntu.com/job/cu2d-choo-choo-ci/394/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/cu2d-choo-choo-ci/394/rebuild

review: Needs Fixing (continuous-integration)

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:868
http://jenkins.qa.ubuntu.com/job/cu2d-choo-choo-ci/395/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/cu2d-choo-choo-ci/395/rebuild

review: Approve (continuous-integration)

[lp:~robru/cupstream2distro/add-ticket-system-support]

869. By Robert Bruce Park on 2014-12-18

Use new Silo merging ticket step.

870. By Robert Bruce Park on 2014-12-18

Ignore exceptions from requests.patch in case ticket system unreachable.

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:869
http://jenkins.qa.ubuntu.com/job/cu2d-choo-choo-ci/396/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/cu2d-choo-choo-ci/396/rebuild

review: Approve (continuous-integration)

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:870
http://jenkins.qa.ubuntu.com/job/cu2d-choo-choo-ci/397/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/cu2d-choo-choo-ci/397/rebuild

review: Approve (continuous-integration)

[lp:~robru/cupstream2distro/add-ticket-system-support]

871. By Robert Bruce Park on 2014-12-18

Refactor set_ticket_status() into patch() for better reusability.

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:871
http://jenkins.qa.ubuntu.com/job/cu2d-choo-choo-ci/398/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/cu2d-choo-choo-ci/398/rebuild

review: Approve (continuous-integration)

[lp:~robru/cupstream2distro/add-ticket-system-support]

872. By Robert Bruce Park on 2014-12-18

HTTP PATCH ticket system once build_ppa is known.

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:872
http://jenkins.qa.ubuntu.com/job/cu2d-choo-choo-ci/399/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/cu2d-choo-choo-ci/399/rebuild

review: Approve (continuous-integration)

[lp:~robru/cupstream2distro/add-ticket-system-support]

873. By Robert Bruce Park on 2014-12-18

Include job_url with step & status info.

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:873
http://jenkins.qa.ubuntu.com/job/cu2d-choo-choo-ci/400/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/cu2d-choo-choo-ci/400/rebuild

review: Approve (continuous-integration)

[Ursula Junque (ursinha)]

Looks good! Thanks for adding the silo PPA and job url, it's more useful now.

review: Approve

[Robert Bruce Park (robru)]

I'm also quite happy with this, I just don't want to merge it (or deploy it to production) until we get a public IP for the ticket system.

review: Approve

[lp:~robru/cupstream2distro/add-ticket-system-support]

874. By Robert Bruce Park on 2014-12-19

Rebase on trunk.

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:874
http://jenkins.qa.ubuntu.com/job/cu2d-choo-choo-ci/405/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/cu2d-choo-choo-ci/405/rebuild

review: Approve (continuous-integration)

[Robert Bruce Park (robru)]

Rejecting because we decided in Austin not to proceed with "CI Engine as Spreadsheet Replacement"

review: Disapprove

Unmerged revisions

874. By Robert Bruce Park on 2014-12-19

Rebase on trunk.

873. By Robert Bruce Park on 2014-12-18

Include job_url with step & status info.

872. By Robert Bruce Park on 2014-12-18

HTTP PATCH ticket system once build_ppa is known.

871. By Robert Bruce Park on 2014-12-18

Refactor set_ticket_status() into patch() for better reusability.

870. By Robert Bruce Park on 2014-12-18

Ignore exceptions from requests.patch in case ticket system unreachable.

869. By Robert Bruce Park on 2014-12-18

Use new Silo merging ticket step.

868. By Robert Bruce Park on 2014-12-18

Rename set_ticket_step to set_ticket_status for consistency.

867. By Robert Bruce Park on 2014-12-18

Drop redundant calls to set_ticket_step().

866. By Robert Bruce Park on 2014-12-18

Tweak variable names.

865. By Robert Bruce Park on 2014-12-18

Better mocks.

=== modified file 'cupstream2distro/silomanager.py'
--- cupstream2distro/silomanager.py	2014-12-08 18:43:01 +0000
+++ cupstream2distro/silomanager.py	2014-12-19 08:25:20 +0000
@@ -19,6 +19,7 @@
 
 import json
 import logging
+import requests
 import os
 import shutil
 import signal
@@ -29,6 +30,7 @@
 
 from cupstream2distro.settings import (
     ALL_SILO_NAMES,
+    SILO_BUILDPPA_SCHEME,
     SILO_NAME_LIST,
     SILO_STATUS_RSYNCDIR,
     SILOS_DIR,
@@ -51,9 +53,27 @@
 SILO_STATE_CLEANING = 11
 
 
+# FIXME: Using bootstack for now, should switch to production once it exists.
+API_URL = 'http://172.19.1.177/api/v1/'
+TICKET_URL = '{}ticket/{{}}/'.format(API_URL)
+
 default_excepthook = sys.excepthook
 
 
+def patch(requestid, data):
+    """Submit an HTTP PATCH to the UCI Engine Ticket System."""
+    try:
+        error = requests.patch(
+            TICKET_URL.format(requestid),
+            headers={'content-type': 'application/json'},
+            data=json.dumps(data)).content
+        if error:
+            raise Exception(error)
+    except Exception as e:
+        logging.error(e.message)
+        return e.message
+
+
 class SiloState(object):
     """This class stores and manipulates the silo config json blob."""
     CONFIG_FILENAME = 'config'
@@ -90,6 +110,8 @@
             'sources': [],
             'mps': {},
         }
+        patch(requestid, dict(build_ppa='ppa:' + SILO_BUILDPPA_SCHEME.format(
+            *siloname.split('/'))))
         return silo_state
 
     @staticmethod
@@ -152,6 +174,13 @@
         logging.debug('Deleting! ' + self.rsyncfile)
         os.unlink(self.rsyncfile)
 
+    def set_ticket_status(self, step, status):
+        """Communicate step and state info to Ticket System."""
+        patch(self.requestid, dict(
+            citrain_overlay=dict(job_url=env.BUILD_URL),
+            current_workflow_step=step,
+            status=status))
+
     def set_config_status(self, state, status, ping=True):
         """Update to reflect the latest status."""
         self._config['global']['status'] = {
@@ -208,46 +237,58 @@
 
     def set_building(self):
         self.set_config_status(SILO_STATE_BUILDING, 'Building')
+        self.set_ticket_status('Silo building', 'In progress')
 
     def set_build_failed(self, message):
         self.set_config_status(SILO_STATE_BUILD_FAILED, message)
+        self.set_ticket_status('Silo building', 'Failed')
 
     def set_build_successful(self):
         self.set_config_status(SILO_STATE_BUILD_SUCCESSFUL, 'Packages built')
+        self.set_ticket_status('Silo building', 'Completed')
 
     def set_publishing(self):
         self.set_config_status(SILO_STATE_PUBLISHING, 'Publishing')
+        self.set_ticket_status('Silo publishing', 'In progress')
 
     def set_publish_failed(self, message):
         self.set_config_status(SILO_STATE_PUBLISH_FAILED,
                                "Can't publish: " + message)
+        self.set_ticket_status('Silo publishing', 'Failed')
 
     def set_cleaning(self, message):
         self.set_config_status(SILO_STATE_CLEANING, message)
+        self.set_ticket_status('Silo merging', 'In progress')
 
     def set_empty(self):
         self.config_step = SILO_EMPTY
 
     def set_merging(self):
         self.set_config_status(SILO_STATE_MERGING, 'Merging')
+        self.set_ticket_status('Silo merging', 'In progress')
 
     def set_merge_failed(self, message):
         self.set_config_status(SILO_STATE_MERGE_FAILED, message)
+        self.set_ticket_status('Silo merging', 'Failed')
 
     def set_migrating(self, message='Packages migrating to destination'):
         self.set_config_status(SILO_STATE_MIGRATING, message)
+        self.set_ticket_status('Silo publishing', 'In progress')
 
     def set_preparing(self):
         self.set_config_status(SILO_STATE_BUILDING, 'Preparing packages')
+        self.set_ticket_status('Silo building', 'In progress')
 
     def set_ready(self):
         self.set_config_status(SILO_STATE_NONE, 'Silo ready to build')
+        self.set_ticket_status('Silo creating', 'Completed')
         self.config_step = SILO_EMPTY
 
     def set_reconfigure_failed(self, message):
         self.config_step = -1
         self.set_config_status(
             SILO_STATE_RECONFIGURE_FAILED, 'Reconfigure failed: ' + message)
+        self.set_ticket_status('Silo creating', 'Failed')
 
     def set_built_checked(self):
         self.config_step = SILO_BUILTCHECKED

=== modified file 'tests/unit/test_silomanager.py'
--- tests/unit/test_silomanager.py	2014-12-08 18:43:01 +0000
+++ tests/unit/test_silomanager.py	2014-12-19 08:25:20 +0000
@@ -2,6 +2,7 @@
 import sys
 import time
 import signal
+import requests
 
 from mock import Mock, patch, call
 from collections import defaultdict
@@ -35,6 +36,8 @@
         self.state._configfile = os_path_join_safe(
             self.data_dir, 'landing-123', 'config')
         self.state.load_config()
+        self.set_ticket_status = self.state.set_ticket_status
+        self.state.set_ticket_status = Mock()
 
     @patch('cupstream2distro.silomanager.SiloState._configfile',
            os_path_join_safe(
@@ -206,7 +209,8 @@
         mkdirMock.assert_called_once_with(
             os.path.expanduser('~/silos/ubuntu-rtm/landing-000'))
 
-    def test_silostate_new_blank(self):
+    @patch('cupstream2distro.silomanager.patch')
+    def test_silostate_new_blank(self, patchMock):
         """Create a new SiloState instance for a fresh silo assignment."""
         silo_state = SiloState.new_blank(
             siloname='ubuntu/landing-789',
@@ -214,6 +218,9 @@
             dest='mars',
             series='parallel',
             landers=['curiosity', 'spirit', 'opportunity'])
+        patchMock.assert_called_once_with(
+            'zippy',
+            {'build_ppa': 'ppa:ci-train-staging-area/ubuntu/landing-789'})
         self.assertEqual(silo_state.siloname, 'ubuntu/landing-789')
         self.assertEqual(
             silo_state.configfile,
@@ -327,24 +334,44 @@
         setMock = self.state.set_config_status = Mock()
         self.state.set_build_successful()
         setMock.assert_called_once_with(3, 'Packages built')
+        self.state.set_ticket_status.assert_called_once_with(
+            'Silo building', 'Completed')
         self.state.set_cleaning('Foo')
         setMock.assert_called_with(11, 'Foo')
+        self.state.set_ticket_status.assert_called_with(
+            'Silo merging', 'In progress')
         self.state.set_merging()
         setMock.assert_called_with(9, 'Merging')
+        self.state.set_ticket_status.assert_called_with(
+            'Silo merging', 'In progress')
         self.state.set_merge_failed('Bar')
         setMock.assert_called_with(10, 'Bar')
+        self.state.set_ticket_status.assert_called_with(
+            'Silo merging', 'Failed')
         self.state.set_migrating('Grill')
         setMock.assert_called_with(8, 'Grill')
+        self.state.set_ticket_status.assert_called_with(
+            'Silo publishing', 'In progress')
         self.state.set_building()
         setMock.assert_called_with(2, 'Building')
+        self.state.set_ticket_status.assert_called_with(
+            'Silo building', 'In progress')
         self.state.set_build_failed('It dun broke')
         setMock.assert_called_with(4, 'It dun broke')
+        self.state.set_ticket_status.assert_called_with(
+            'Silo building', 'Failed')
         self.state.set_preparing()
         setMock.assert_called_with(2, 'Preparing packages')
+        self.state.set_ticket_status.assert_called_with(
+            'Silo building', 'In progress')
         self.state.set_publishing()
         setMock.assert_called_with(6, 'Publishing')
+        self.state.set_ticket_status.assert_called_with(
+            'Silo publishing', 'In progress')
         self.state.set_publish_failed('Doh!')
         setMock.assert_called_with(7, "Can't publish: Doh!")
+        self.state.set_ticket_status.assert_called_with(
+            'Silo publishing', 'Failed')
 
     def test_silostate_step_setters(self):
         """Set various silo steps."""
@@ -356,6 +383,7 @@
         self.assertEqual(self.state._config['global']['step'], 2)
         self.state.set_done()
         self.assertEqual(self.state._config['global']['step'], 3)
+        self.assertEqual(self.state.set_ticket_status.call_count, 0)
 
     def test_silostate_getters(self):
         """Read various state properties."""
@@ -450,6 +478,8 @@
         self.state.set_ready()
         statusMock.assert_called_once_with(1, 'Silo ready to build')
         self.assertEqual(self.state.config_step, 0)
+        self.state.set_ticket_status.assert_called_once_with(
+            'Silo creating', 'Completed')
 
     def test_silostate_set_reconfigure_failed(self):
         """Set reconfigure failed status."""
@@ -458,6 +488,8 @@
         statusMock.assert_called_once_with(
             5, 'Reconfigure failed: You did a bad thing.')
         self.assertEqual(self.state.config_step, -1)
+        self.state.set_ticket_status.assert_called_once_with(
+            'Silo creating', 'Failed')
 
     def test_silostate_append_mp(self):
         """Add an MP to the list of that project's MPs."""
@@ -514,6 +546,57 @@
         self.assertEqual(
             self.state._config['additional_targets'], ['a', 'b', 'c'])
 
+    @patch('cupstream2distro.silomanager.json')
+    @patch('cupstream2distro.silomanager.requests')
+    def test_set_ticket_status(self, requestsMock, jsonMock):
+        """Report statuses to the ticket system correctly."""
+        env.BUILD_URL = 'http://example.com/zoinks'
+        requestsMock.patch.return_value.content = ''
+        self.set_ticket_status('Hi', 'there')
+        jsonMock.dumps.assert_called_once_with(
+            dict(current_workflow_step='Hi',
+                 status='there',
+                 citrain_overlay=dict(job_url='http://example.com/zoinks')))
+        requestsMock.patch.assert_called_once_with(
+            'http://172.19.1.177/api/v1/ticket/1411123362781/',
+            headers={'content-type': 'application/json'},
+            data=jsonMock.dumps())
+
+    @patch('cupstream2distro.silomanager.logging')
+    @patch('cupstream2distro.silomanager.json')
+    @patch('cupstream2distro.silomanager.requests')
+    def test_set_ticket_status_fail(self, requestsMock, jsonMock, logMock):
+        """Report errors from the ticket system correctly."""
+        env.BUILD_URL = 'http://example.com/zoinks'
+        requestsMock.patch.return_value.content = '{"errors":"you fail"}'
+        self.set_ticket_status('Hi', 'there')
+        jsonMock.dumps.assert_called_once_with(
+            dict(current_workflow_step='Hi',
+                 status='there',
+                 citrain_overlay=dict(job_url='http://example.com/zoinks')))
+        logMock.error.assert_called_once_with(
+            '{"errors":"you fail"}')
+        requestsMock.patch.assert_called_once_with(
+            'http://172.19.1.177/api/v1/ticket/1411123362781/',
+            headers={'content-type': 'application/json'},
+            data=jsonMock.dumps())
+
+    @patch('cupstream2distro.silomanager.logging')
+    @patch('cupstream2distro.silomanager.json')
+    @patch('cupstream2distro.silomanager.requests')
+    def test_set_ticket_status_except(self, requestsMock, jsonMock, logMock):
+        """Report errors from the requests system correctly."""
+        requestsMock.patch.return_value.content = ''
+        requestsMock.patch.side_effect = requests.ConnectionError(
+            '[Errno 110] Connection timed out')
+        self.set_ticket_status('Hi', 'there')
+        logMock.error.assert_called_once_with(
+            '[Errno 110] Connection timed out')
+        requestsMock.patch.assert_called_once_with(
+            'http://172.19.1.177/api/v1/ticket/1411123362781/',
+            headers={'content-type': 'application/json'},
+            data=jsonMock.dumps())
+
     def test_signal_handler(self):
         """Our signal handler is installed globally."""
         self.assertTrue(