Merge lp:~cjohnston/uci-engine/ts-amulet-settle into lp:uci-engine

[Chris Johnston]

Add more deployment padding for the ticket_system amulet tests

Currently the ticket system amulet tests are failing almost 100% of the time because the tests are being run prior to the deployment having time to settle. If we add a get_ip_and_port for the ts-django instance, it reduces the frequency of the failures, as get_ip_and_port will wait for the ts-django port to become available. Also adds a note to the failing test that it is flaky.

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:489
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/637/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/637/rebuild

review: Approve (continuous-integration)

[Andy Doan (doanac)]

Could we just put a single call to "get_ip_and_port('ts-django')" in the
setUp method?

[lp:~cjohnston/uci-engine/ts-amulet-settle]

489. By Chris Johnston on 2014-05-15

[r=Francis Ginther, PS Jenkins bot] Add missing import checks to tests/run.py from Chris Johnston

490. By Andy Doan on 2014-05-15

[r=Francis Ginther, PS Jenkins bot] bsbuilder: better throttling of the status updates

We are flooding the log with the same status message over
and over. This helps throttle this back a bit so that we
still see updates and can tell things aren't stuck, but also
not flood out log files. I added this in a generic way
to amqp_utils so that other people can choose to join in
if they ever find the need. from Andy Doan

491. By Chris Johnston on 2014-05-16

[r=Andy Doan, PS Jenkins bot] Remove deps like python-django from needing to be installed in order to create a ticket. 1319911 from Chris Johnston

[Vincent Ladeuil (vila)]

+1 on Andy's suggestion, no need to duplicate the call and it will be clearer in the setUp().

Oh there is no setup yet ! Don't forget to call the base class setup there then:

def setUp(self):
    super(TicketSystemTest, self.).setUp()
    # We need to ensure that both webui-apache and ts-django are up and running while we wait
    # for a fix for http://pad.lv/1319077
    self.get_ip_and_port('ts-django')
    self.get_ip_and_port('webui-apache')

or something similar (untested).

Thanks for your patience and work there.

review: Needs Fixing

[lp:~cjohnston/uci-engine/ts-amulet-settle]

492. By Vincent Ladeuil on 2014-05-16

[r=PS Jenkins bot, Celso Providelo] Silent scripts. from Vincent Ladeuil

493. By Celso Providelo on 2014-05-16

[r=PS Jenkins bot, Vincent Ladeuil] Extending Gatekeeper SourcePackageUpload validation to cope with re-raised dput exceptions. from Celso Providelo

494. By Andy Doan on 2014-05-19

[r=Paul Larson, PS Jenkins bot] django charm: add relationship support for rabbitmq

This lets you add a relation between rabbitmq and django like we
do in our rabbitmq-worker charm. This allows the ticket system to use rabbit from Andy Doan

495. By Vincent Ladeuil on 2014-05-20

[r=Evan Dandrea, PS Jenkins bot] Fix the tests that didn't catch the broken call nor the broken assumptions about rabbit behavior. Also fix the broken basic_publish() call. 1320000 from Vincent Ladeuil

496. By Andy Doan on 2014-05-20

[r=Celso Providelo, PS Jenkins bot] lander/ticket-system: convert to rabbit

This changes the lander/ticket relationship so that the ticket
system simply sends amqp messages that the lander can handle. This
is the first step in allowing us to scale to multiple landers. from Andy Doan

497. By Andy Doan on 2014-05-21

[r=Chris Johnston, PS Jenkins bot] remove incorrect comment about ticket-states

this comment is incorrect and just leads to confusion from Andy Doan

498. By Andy Doan on 2014-05-21

[r=Chris Johnston, PS Jenkins bot] ticket-system: make test_style automatically ignore migrations

Its a pain to remember adding your migration to this file. This
discovers them for you.
  from Andy Doan

499. By Andy Doan on 2014-05-21

[r=Vincent Ladeuil, PS Jenkins bot] webui: allow view ticket progress live

This is a super simplistic start, but allows you to follow the progress of a ticket landing in real time. from Andy Doan

500. By Vincent Ladeuil on 2014-05-21

[r=PS Jenkins bot, Andy Doan] Uprgade to ucitests 0.1.3 and remove the corresponding upstreamed code. from Vincent Ladeuil

501. By Ursula Junque on 2014-05-21

[r=Andy Doan, PS Jenkins bot, Celso Providelo] This branch adds the publisher skeleton to our tree, also adds PUBLISHER_QUEUE type to amqp_utils.py from Ursula Junque

502. By Andy Doan on 2014-05-22

[r=PS Jenkins bot, Ursula Junque, Chris Johnston] ticket-system: fix series choices for tickets

Things have been working by dumb luck because we were creating tickets
via the REST api with a series in all lowercase (ie "saucy").

I was playing with editing tickets from the django admin panel and
discovered we saved the series as SAUCY. This breaks the bsbuilder's
interaction with the LP api which expectes a series to be all lowercase.

I fixed that, removed the obsolete quantal series and included trusty. from Andy Doan

503. By Andy Doan on 2014-05-22

[r=Francis Ginther, PS Jenkins bot] ticket-system: allow updating of just status or workflow-step

This fixes a bug and annoyance in the ticket system.

bug - you can't just update the status or workflow-step. They have
to both be provided

annoyance - we return status and workflow step back as words like
"in progress", but our write-api requires the inputs to be numerical.

This fixes both issues so that you can now just update a ticket's status
to something like 'failed'.
  from Andy Doan

504. By Evan on 2014-05-22

[r=PS Jenkins bot, Paul Larson] Put the right data into the lp_creds.txt file. from Evan Dandrea

505. By Celso Providelo on 2014-05-23

[r=PS Jenkins bot] Updating restish charm according to IS guidelines. from Celso Providelo

506. By Celso Providelo on 2014-05-23

[r=PS Jenkins bot, Evan Dandrea] Updating our charms to cope with prodstack guidelines from Celso Providelo

507. By Chris Johnston on 2014-05-26

[r=PS Jenkins bot, Andy Doan] Add restish charm to local charms in deploy.py from Chris Johnston

508. By Chris Johnston on 2014-05-26

Add more deployment padding for the ticket_system amulet tests

[PS Jenkins bot (ps-jenkins)]

PASSED: Continuous integration, rev:509
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/699/
Executed test runs:

Click here to trigger a rebuild:
http://s-jenkins.ubuntu-ci:8080/job/uci-engine-ci/699/rebuild

review: Approve (continuous-integration)

[lp:~cjohnston/uci-engine/ts-amulet-settle]

509. By Chris Johnston on 2014-05-26

Move checks to setUp

[Chris Johnston (cjohnston)]

Addressed comments

review: Needs Resubmitting

[Vincent Ladeuil (vila)]

Progress !

review: Approve

=== modified file 'tests/deployers.py'
--- tests/deployers.py	2014-05-13 15:54:55 +0000
+++ tests/deployers.py	2014-05-26 12:52:06 +0000
@@ -71,7 +71,7 @@
         self.deployer.load(script)
         self.addCleanup(self.deployer.tearDown)
         self.deployer.setUp(self.timeout)
-        self.status =  self.deployer.status()
+        self.status = self.deployer.status()
 
     def fixup_includes(self, script, base_dir):
         # amulet can only process "include-base64" option if the file is under
@@ -103,7 +103,7 @@
                 break
             except KeyError:
                 tries += 1
-                log.warn("open-ports not found waiting...")
+                log.warn("open-ports not found for %s, waiting..." % service)
                 time.sleep(5)
 
     def assert_job_running(self, upstart_job, service, unit=0):

=== modified file 'tests/test_ticket_system.py'
--- tests/test_ticket_system.py	2014-05-07 18:18:29 +0000
+++ tests/test_ticket_system.py	2014-05-26 12:52:06 +0000
@@ -27,6 +27,13 @@
 
     deployer_cfg = 'ticket-system.yaml'
 
+    def setUp(self):
+        super(TicketSystemTest, self).setUp()
+        # Adding get_ip_and_port('ts-django') in order to allow the deployment
+        # further time to settle before running the tests.
+        # http://pad.lv/1319077
+        self.get_ip_and_port('ts-django')
+
     def get_server_status_and_content(self, url):
         final_url = url.format(self.get_ip_and_port('webui-apache')[0])
 
@@ -51,6 +58,7 @@
 
     def test_ticket_sys_api(self):
         url = 'http://{}/api/v1/ticket'
+        self.get_ip_and_port('ts-django')
         resp, content = self.get_server_status_and_content(url)
 
         self.assertEqual('200', resp['status'])
@@ -63,7 +71,9 @@
                           "added_binaries": "binary-1,binary-2",
                           "removed_binaries": "binary-3,binary-4"}
         resp, content = self.create_ticket(ticket_content)
-        self.assertEqual('201', resp['status'])
+        self.assertEqual('201', resp['status'], 'This is a known flaky test. '
+                         'In order to fix it we need to fix '
+                         'http://pad.lv/1319077')
 
     def test_prohibit_delete(self):
         ticket_content = {'owner': "[e-mail]",