Ubuntu CI Services

Merge lp:~vila/ubuntu-ci-services-itself/testy into lp:ubuntu-ci-services-itself

  1. testy
  2. Merge into trunk
Proposed by Vincent Ladeuil
Status: Merged
Approved by: Francis Ginther
Approved revision: no longer in the source branch.
Merged at revision: 14
Proposed branch: lp:~vila/ubuntu-ci-services-itself/testy
Merge into: lp:ubuntu-ci-services-itself
Diff against target: 402 lines (+112/-56)
8 files modified
docs/components/existing-pieces.rst (+1/-1)
docs/components/landing-manager.rst (+4/-4)
docs/components/planned.rst (+1/-1)
docs/components/test-runner.rst (+75/-19)
docs/components/ticket-manager.rst (+3/-3)
docs/components/ticket-system.rst (+1/-1)
docs/sequence.rst (+23/-23)
docs/timeline.rst (+4/-4)
To merge this branch: bzr merge lp:~vila/ubuntu-ci-services-itself/testy
Reviewer Review Type Date Requested Status
Francis Ginther Approve
Review via email: mp+198083@code.launchpad.net

Commit message

Update the Test Runner component design and provide the initial API.

Description of the change

Main changes from the trunk version:

* 'test_image' request

  s/test_image(image_url, packages, archive_id, progress_trigger)
   /test_image(test_request_id, image_url, package)

  The rationale is that the caller should do multiple requests if it needs
  multiple test results (otherwise the 'done' request will have to aggregate
  the results and the artifacts and the caller will have to sort them out =>
  unneeded complexity).

* handles a single DEP8 package which produce a single test result and a
  list of artifacts (phase0). For following phases, I'd like to embed the
  articats in the test result itself as discussed during the 'subunit'
  hangout. For now, the request will contain them all, leaving the caller
  store them as it sees fit.

  The rationale is that asking this component to know about the data store
  is coupling them for no good reason.

* new requests: 'done' and 'progress'

Questions:

* messages exchanged between components

We talked about REST APIs and about task queues. I more or less assumed that
a request is added to a queue as a task and the response is sent back when
the task is done (from the queue POV). Feedback welcome.

For the 'done' request, the above works ok, but if we want a REST API and no
queue, I need to know *where* to send that request (some url provided as
part of the 'test_package' request parameters ?).

* progress

Emitting progress and providing an ETA or some estimate of the work already
done raise two issues:

- how many messages should be sent or alternatively at which rate ? Emitting
  the same number of messages for a 1 minute job and a several hours one
  doesn't make much sense. It seems to me that one message very minute or
  every 30 seconds should be enough. But at scale, that may still be far too
  much messages. Thoughts ?

- ETA is a pipe dream so let's settle for estimates to start with. This can
  be be based on the number of tests as long as each test is shorter than
  the interval between two messages. If it's not, we risk misreadings
  between a long test and a hanging test.

If we don't get good ideas there, I'd settle with some arbitrary interval
for phase 0 and plan to acquire the number of tests to initialize 'total'.

To post a comment you must log in.
Revision history for this message
Andy Doan (doanac) wrote :

On 12/06/2013 09:34 AM, Vincent Ladeuil wrote:
> + test_image(image_url, package_name) -> test_request_id

> +This ends the processing of a 'done' request and is sent to the 'Landing
> +Manager'.

> + done(test_request_id, status=[FAIL, SUCCESS], test_result, artifacts)

This introduces a new decision for us to make. In the
branch-source-builder we have something called a "progress_trigger". It
appears to be the same concept as your "test_request_id". So the first
decision is simply agreeing on a common name for this thing.

The thing where the branch-source-builder really differs is on who
creates progress_trigger. In its design, the caller passes this value.
In your design the implementation creates and returns the value. I don't
have a strong opinion. However, I suspect the branch-source-builder's
approach will be more HA friendly. eg, it can create this object and
persist it somehow for the service. If the calling service dies while
the request is being processed, its okay. The messages will get queued
up somewhere to be determined, and when the back-up service comes back
online it will know about this progress_trigger and be able to catch
back up with the queued responses.

Revision history for this message
Francis Ginther (fginther) wrote :

Thanks for agreeing on the caller/callee semantics for passing of the test_request_id as the queue handle from the Lander to the Test Runner.

This MP looks good, I'll approve.

The questions you raise above merit their own MP. I'll hopefully get to a proposal by Monday morning. The general idea is to:
 - define the calling convention to pass the progress queue handle from the caller to the callee (as Andy described)
 - define progress semantics (what is the progress interval, what is considered a timeout, role of caller and callee)
 - define task queue semantics (role of the producer and consumer)

review: Approve
lp:~vila/ubuntu-ci-services-itself/testy updated
11. By Andy Doan

[r=Francis Ginther, Evan Dandrea] updated branch/source builder design from Andy Doan

12. By Andy Doan

[r=Francis Ginther] adds first pass of ppa-assigner

more changes to come following design updates. from Andy Doan

13. By Andy Doan

[r=Francis Ginther, Evan Dandrea] updated branch/source builder design from Andy Doan

14. By Vincent Ladeuil

[r=Francis Ginther] Update the Test Runner component design and provide the initial API. from Vincent Ladeuil

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'docs/components/existing-pieces.rst'
--- docs/components/existing-pieces.rst 2013-11-16 10:12:08 +0000
+++ docs/components/existing-pieces.rst 2013-12-06 17:26:10 +0000
@@ -18,7 +18,7 @@
1818
19* Project specific configuration - We are doing this now with lp:cupstream2distro-config but with a very jenkins focused implementation.19* Project specific configuration - We are doing this now with lp:cupstream2distro-config but with a very jenkins focused implementation.
2020
21**Integration Test Service**21**Test Runner**
2222
23* Test full touch images - lp:ubuntu-test-cases/touch?23* Test full touch images - lp:ubuntu-test-cases/touch?
24* Test custom touch images based on ppas and packages - lp:ubuntu-test-cases/touch (? need to ask doanac/plars)24* Test custom touch images based on ppas and packages - lp:ubuntu-test-cases/touch (? need to ask doanac/plars)
2525
=== modified file 'docs/components/landing-manager.rst'
--- docs/components/landing-manager.rst 2013-11-25 23:32:31 +0000
+++ docs/components/landing-manager.rst 2013-12-06 17:26:10 +0000
@@ -8,8 +8,8 @@
8*Deployment:*8*Deployment:*
99
10* Can run as a juju service.10* Can run as a juju service.
11* Needs relationship to Project Manager, Branch/Source Builder and Integration Test Runner.11* Needs relationship to Project Manager, Branch/Source Builder and Test Runner.
12* Shouldn’t need to save state, should be able to recreate state upon restart. This can be done by requesting the general state from the Project Manager and then refining by looking at the MP, the PPA, the Integration Test Runner and the Archive.12* Shouldn’t need to save state, should be able to recreate state upon restart. This can be done by requesting the general state from the Project Manager and then refining by looking at the MP, the PPA, the Test Runner and the Archive.
13* No public access needed.13* No public access needed.
1414
15*Interactions:*15*Interactions:*
@@ -19,7 +19,7 @@
19* Branch/Source Builder - MP branches are dispatched to the builder.19* Branch/Source Builder - MP branches are dispatched to the builder.
20* PPA - Location where the package build will take place.20* PPA - Location where the package build will take place.
21* PPA Assigneer - Provides a single PPA to perform package builds.21* PPA Assigneer - Provides a single PPA to perform package builds.
22* Integration Test Runner - Runs the specified test (if any) on the MP packages from the PPA.22* Test Runner - Runs the specified test (if any) on the MP packages from the PPA.
23* Archive - The packages are copied to the archive from the PPA on successful completion of testing.23* Archive - The packages are copied to the archive from the PPA on successful completion of testing.
2424
25*Development Plan:*25*Development Plan:*
@@ -30,7 +30,7 @@
30* Allocate a PPA (interact with PPA Assigner).30* Allocate a PPA (interact with PPA Assigner).
31* Monitor a PPA for build status.31* Monitor a PPA for build status.
32* Report status (interact with Project Manager).32* Report status (interact with Project Manager).
33* Initiate integration tests (interact with Integration Test Service).33* Initiate integration tests (interact with Test Runner).
34* Ensure that trunk has not changed during execution.34* Ensure that trunk has not changed during execution.
35* Merge an MP to trunk.35* Merge an MP to trunk.
36* Binary copy a package from PPA to the archive.36* Binary copy a package from PPA to the archive.
3737
=== modified file 'docs/components/planned.rst'
--- docs/components/planned.rst 2013-11-25 23:32:31 +0000
+++ docs/components/planned.rst 2013-12-06 17:26:10 +0000
@@ -12,4 +12,4 @@
12 landing-manager12 landing-manager
13 branch-source-builder13 branch-source-builder
14 image-builder14 image-builder
15 integration-test-service15 test-runner
1616
=== renamed file 'docs/components/integration-test-service.rst' => 'docs/components/test-runner.rst'
--- docs/components/integration-test-service.rst 2013-11-16 10:12:08 +0000
+++ docs/components/test-runner.rst 2013-12-06 17:26:10 +0000
@@ -1,22 +1,78 @@
1Integration Test Service1===========
2========================2Test Runner
33===========
4*Purpose:*4
55Purpose
6* Performs testing given some mix of packages and PPAs, or an image and tests to run.6=======
77
8*Deployment:*8* Performs testing given some mix of packages and PPAs, or an image and
99 tests to run.
10* This can deployed as a juju service, but the actual test runners may be on bare metal.10
11* This provides the slave service indicated here: https://drive.google.com/a/canonical.com/?usp=chrome_app#folders/0BxYc9b_WOJIYYlJFejl2a3FWTVU11API for submitting test run requests
1212====================================
13*Interactions:*13
1414The 'Test Runner' exchanges requests/responses with the 'Landing Manager'.
15* Trunk Delivering System - Supplies test requests and waits for results.15
1616test_package
17*Development Plan:*17------------
1818
19* Define API for submitting test requests.19This execute the DEB8 'package' on 'image_url' and is sent by the
20'Landing Manager'.
21
22.. code-block:: none
23
24 test_image(test_request_id, image_url, package_name)
25
26The caller is responsible for providing a unique 'test_request_id' identifier.
27
28This is composed of:
29 * download an image from 'image_url'
30 * boots from it
31 * intall 'package_name' monitoring its execution emitting 'progress'
32 requests (see below).
33 * collect the artifacts
34 * send a 'done' request
35
36done
37----
38
39This ends the processing of a 'done' request and is sent to the 'Landing
40Manager'.
41
42.. code-block:: none
43
44 done(test_request_id, status=[FAIL, SUCCESS], test_result, artifacts)
45
46
47progress
48--------
49
50This is sent to the 'Landing Manager' to provide feedback during the test run.
51
52.. code-block:: none
53
54 progress(test_request_id, message='% or ETA', current, total)
55
56
57Based on test execution feedback, a progress message is sent at regular
58intervals until completion. 'total' is the number of tests to execute,
59'current' is the rank of the test currently running. Both can be empty if
60this info is not available.
61
62Deployment
63==========
64
65* This can be deployed as a juju service, but the actual test runners may be on
66 bare metal (vms only for phase 0).
67
68Interactions
69============
70
71* Landing Manager - Supplies test requests and waits for results.
72
73Development Plan
74================
75
20* Separate environment setup and test runs76* Separate environment setup and test runs
21* Schedule a primitive test.77* Schedule a primitive test.
22* Perform an autopilot test on a touch device.78* Perform an autopilot test on a touch device.
2379
=== modified file 'docs/components/ticket-manager.rst'
--- docs/components/ticket-manager.rst 2013-11-25 23:32:31 +0000
+++ docs/components/ticket-manager.rst 2013-12-06 17:26:10 +0000
@@ -19,7 +19,7 @@
19*Deployment:*19*Deployment:*
2020
21* Can run as a juju service.21* Can run as a juju service.
22* Needs relationship to Project Manager, Branch Listener, Branch/Source Builder, Image Builder, Integration Test Runner and the Landing Manager.22* Needs relationship to Project Manager, Branch Listener, Branch/Source Builder, Image Builder, Test Runner and the Landing Manager.
23* Shouldn’t need to save state, should be able to recreate state upon restart.23* Shouldn’t need to save state, should be able to recreate state upon restart.
24* No public access needed.24* No public access needed.
2525
@@ -31,7 +31,7 @@
31* Branch/Source Builder - MP branches are dispatched to the builder.31* Branch/Source Builder - MP branches are dispatched to the builder.
32* PPAs - Two PPAs are used for performing package builds.32* PPAs - Two PPAs are used for performing package builds.
33* Image Builder - Generates up to 3 images to be used for testing: feature branches + base image, feature branches + latest image, feature branches + trunk + latest image.33* Image Builder - Generates up to 3 images to be used for testing: feature branches + base image, feature branches + latest image, feature branches + trunk + latest image.
34* Integration Test Runner - Runs the specified tests on up to 3 images.34* Test Runner - Runs the specified tests on up to 3 images.
3535
36*Development Plan:*36*Development Plan:*
3737
@@ -44,7 +44,7 @@
44* Report status (interact with Project Manager).44* Report status (interact with Project Manager).
45* Return PPAs.45* Return PPAs.
46* Build an image (interact with Image Builder).46* Build an image (interact with Image Builder).
47* Initiate integration tests (interact with Integration Test Service).47* Initiate integration tests (interact with Test Runner).
48* Implement the "Land Now" button. Dispatch individual trunk merge requests (interact with Landing Manager).48* Implement the "Land Now" button. Dispatch individual trunk merge requests (interact with Landing Manager).
49* Full end-to-end “Workflow II A: feature branch/transitions/fix involving multiple components” success case.49* Full end-to-end “Workflow II A: feature branch/transitions/fix involving multiple components” success case.
50* Handle source build failures.50* Handle source build failures.
5151
=== modified file 'docs/components/ticket-system.rst'
--- docs/components/ticket-system.rst 2013-11-25 23:32:31 +0000
+++ docs/components/ticket-system.rst 2013-12-06 17:26:10 +0000
@@ -33,6 +33,6 @@
3333
34* track MPs targeting feature branches34* track MPs targeting feature branches
35* track packages built as a result of the MP/development done (PPA? Archive?)35* track packages built as a result of the MP/development done (PPA? Archive?)
36* integration test runner for both environments (the isolated one and the automatic merges against the currently landed)36* test runner for both environments (the isolated one and the automatic merges against the currently landed)
3737
3838
3939
=== modified file 'docs/sequence.rst'
--- docs/sequence.rst 2013-11-16 17:37:27 +0000
+++ docs/sequence.rst 2013-12-06 17:26:10 +0000
@@ -22,7 +22,7 @@
22 * Branch/Source builder22 * Branch/Source builder
23 * ppa assigner23 * ppa assigner
24 * Image builder24 * Image builder
25 * Integration test runner25 * Test runner
2626
27===============27===============
28Delivery system28Delivery system
@@ -141,7 +141,7 @@
141| participant Branch/Source builder141| participant Branch/Source builder
142| participant ppa assigner142| participant ppa assigner
143| participant ppa143| participant ppa
144| participant Integration test runner144| participant Test runner
145| participant archive145| participant archive
146146
147| Note over Projects manager: new ticket in the queue147| Note over Projects manager: new ticket in the queue
@@ -158,8 +158,8 @@
158| Note over Landing manager:report of successful build158| Note over Landing manager:report of successful build
159| Landing manager->Projects manager: update status159| Landing manager->Projects manager: update status
160| Projects manager->Landing manager:get possible overrides160| Projects manager->Landing manager:get possible overrides
161| Landing manager->Integration test runner: test against latest proposed image161| Landing manager->Test runner: test against latest proposed image
162| Integration test runner->Landing manager: results162| Test runner->Landing manager: results
163| Landing manager->Projects manager: update status163| Landing manager->Projects manager: update status
164| Projects manager->Landing manager:get possible overrides164| Projects manager->Landing manager:get possible overrides
165| Landing manager-->archive: copy source from ppa to archive and monitor the transition165| Landing manager-->archive: copy source from ppa to archive and monitor the transition
@@ -177,7 +177,7 @@
177| participant Branch/Source builder177| participant Branch/Source builder
178| participant ppa assigner178| participant ppa assigner
179| participant ppa179| participant ppa
180| participant Integration test runner180| participant Test runner
181| participant archive181| participant archive
182182
183| Note over Projects manager: new ticket in the queue183| Note over Projects manager: new ticket in the queue
@@ -199,7 +199,7 @@
199| participant Branch/Source builder199| participant Branch/Source builder
200| participant ppa assigner200| participant ppa assigner
201| participant ppa201| participant ppa
202| participant Integration test runner202| participant Test runner
203| participant archive203| participant archive
204204
205| Note over Projects manager: new ticket in the queue205| Note over Projects manager: new ticket in the queue
@@ -225,7 +225,7 @@
225| participant Branch/Source builder225| participant Branch/Source builder
226| participant ppa assigner226| participant ppa assigner
227| participant ppa227| participant ppa
228| participant Integration test runner228| participant Test runner
229| participant archive229| participant archive
230230
231| Note over Projects manager: new ticket in the queue231| Note over Projects manager: new ticket in the queue
@@ -242,8 +242,8 @@
242| Note over Landing manager:report of successful build242| Note over Landing manager:report of successful build
243| Landing manager->Projects manager: update status243| Landing manager->Projects manager: update status
244| Projects manager->Landing manager:get possible overrides244| Projects manager->Landing manager:get possible overrides
245| Landing manager->Integration test runner: test against latest proposed image and FAILED245| Landing manager->Test runner: test against latest proposed image and FAILED
246| Integration test runner->Landing manager: failure results246| Test runner->Landing manager: failure results
247| Landing manager->ppa assigner: deprovision that ppa247| Landing manager->ppa assigner: deprovision that ppa
248| Landing manager->Projects manager: update failure248| Landing manager->Projects manager: update failure
249| Note over Projects manager: ticket removed from the queue249| Note over Projects manager: ticket removed from the queue
@@ -256,7 +256,7 @@
256| participant Branch/Source builder256| participant Branch/Source builder
257| participant ppa assigner257| participant ppa assigner
258| participant ppa258| participant ppa
259| participant Integration test runner259| participant Test runner
260| participant archive260| participant archive
261 261
262| Note over Projects manager: new ticket in the queue262| Note over Projects manager: new ticket in the queue
@@ -273,8 +273,8 @@
273| Note over Landing manager:report of successful build273| Note over Landing manager:report of successful build
274| Landing manager->Projects manager: update status274| Landing manager->Projects manager: update status
275| Projects manager->Landing manager:get possible overrides275| Projects manager->Landing manager:get possible overrides
276| Landing manager->Integration test runner: test against latest proposed image276| Landing manager->Test runner: test against latest proposed image
277| Integration test runner->Landing manager: results277| Test runner->Landing manager: results
278| Landing manager->Projects manager: update status278| Landing manager->Projects manager: update status
279| Projects manager->Landing manager:get possible overrides279| Projects manager->Landing manager:get possible overrides
280| Landing manager-->archive: copy source from ppa to archive and monitor the transition280| Landing manager-->archive: copy source from ppa to archive and monitor the transition
@@ -298,7 +298,7 @@
298| participant Branch/Source builder298| participant Branch/Source builder
299| participant ppa assigner299| participant ppa assigner
300| participant ppa300| participant ppa
301| participant Integration test runner301| participant Test runner
302| participant archive302| participant archive
303303
304| Note over Projects manager: new ticket in the queue304| Note over Projects manager: new ticket in the queue
@@ -315,8 +315,8 @@
315| Note over Landing manager:report of successful build315| Note over Landing manager:report of successful build
316| Landing manager->Projects manager: update status316| Landing manager->Projects manager: update status
317| Projects manager->Landing manager:get possible overrides317| Projects manager->Landing manager:get possible overrides
318| Landing manager->Integration test runner: test against latest proposed image318| Landing manager->Test runner: test against latest proposed image
319| Integration test runner->Landing manager: results319| Test runner->Landing manager: results
320| Landing manager->Projects manager: update status320| Landing manager->Projects manager: update status
321| Projects manager->Landing manager:get possible overrides321| Projects manager->Landing manager:get possible overrides
322| Landing manager-->archive: copy source from ppa to archive and monitor the transition322| Landing manager-->archive: copy source from ppa to archive and monitor the transition
@@ -335,8 +335,8 @@
335| Note over Landing manager:report of successful build335| Note over Landing manager:report of successful build
336| Landing manager->Projects manager: update status336| Landing manager->Projects manager: update status
337| Projects manager->Landing manager:get possible overrides337| Projects manager->Landing manager:get possible overrides
338| Landing manager->Integration test runner: test against latest proposed image338| Landing manager->Test runner: test against latest proposed image
339| Integration test runner->Landing manager: results339| Test runner->Landing manager: results
340| Landing manager->Projects manager: update status340| Landing manager->Projects manager: update status
341| Projects manager->Landing manager:get possible overrides341| Projects manager->Landing manager:get possible overrides
342| Landing manager-->archive: copy source from ppa to archive and monitor the transition342| Landing manager-->archive: copy source from ppa to archive and monitor the transition
@@ -354,7 +354,7 @@
354| participant Branch/Source builder354| participant Branch/Source builder
355| participant ppa assigner355| participant ppa assigner
356| participant ppa356| participant ppa
357| participant Integration test runner357| participant Test runner
358| participant archive358| participant archive
359359
360| Note over Projects manager: new ticket in the queue360| Note over Projects manager: new ticket in the queue
@@ -371,8 +371,8 @@
371| Note over Landing manager:report of successful build371| Note over Landing manager:report of successful build
372| Landing manager->Projects manager: update status372| Landing manager->Projects manager: update status
373| Projects manager->Landing manager:get possible overrides373| Projects manager->Landing manager:get possible overrides
374| Landing manager->Integration test runner: test against latest proposed image374| Landing manager->Test runner: test against latest proposed image
375| Integration test runner->Landing manager: results375| Test runner->Landing manager: results
376| Landing manager->Projects manager: update status376| Landing manager->Projects manager: update status
377| Projects manager->Landing manager:get possible overrides377| Projects manager->Landing manager:get possible overrides
378| Landing manager-->archive: copy source from ppa to archive and monitor the transition378| Landing manager-->archive: copy source from ppa to archive and monitor the transition
@@ -433,7 +433,7 @@
433| participant Branch/Source builder (trunk + branch)433| participant Branch/Source builder (trunk + branch)
434| participant ppa (branch)434| participant ppa (branch)
435| participant ppa (trunk + branch)435| participant ppa (trunk + branch)
436| participant Integration test runner436| participant Test runner
437437
438| Note over Projects manager: new ticket opened, specifying components438| Note over Projects manager: new ticket opened, specifying components
439| Projects manager->Ticket manager: Assign branch responsibility get metadata (like integration tests to run)439| Projects manager->Ticket manager: Assign branch responsibility get metadata (like integration tests to run)
@@ -477,8 +477,8 @@
477| Image builder-->ppa (trunk + branch): build IMAGE C (trunk merged with latest image num)477| Image builder-->ppa (trunk + branch): build IMAGE C (trunk merged with latest image num)
478| Image builder->Ticket manager: report and publish isos478| Image builder->Ticket manager: report and publish isos
479| Ticket manager->Projects manager: push infos479| Ticket manager->Projects manager: push infos
480| Ticket manager->Integration test runner: test against available images480| Ticket manager->Test runner: test against available images
481| Integration test runner->Ticket manager: results481| Test runner->Ticket manager: results
482| Ticket manager->Projects manager: update status482| Ticket manager->Projects manager: update status
483| Note left of Ticket manager: END TESTING483| Note left of Ticket manager: END TESTING
484484
485485
=== modified file 'docs/timeline.rst'
--- docs/timeline.rst 2013-11-16 10:12:08 +0000
+++ docs/timeline.rst 2013-12-06 17:26:10 +0000
@@ -10,7 +10,7 @@
10=============================10=============================
11spec: https://docs.google.com/a/canonical.com/document/d/1GUuASKaXVoWpVi4WG54meW5bMibEzXLVeRCnxjhmtQg/edit#11spec: https://docs.google.com/a/canonical.com/document/d/1GUuASKaXVoWpVi4WG54meW5bMibEzXLVeRCnxjhmtQg/edit#
1212
13* Integration test runner (unify upstream merger and cu2d and smoke?)13* Test runner (unify upstream merger and cu2d and smoke?)
14* launchpad monitor (health check, track merge proposals, package builds)14* launchpad monitor (health check, track merge proposals, package builds)
15* device manager (flashing, upgrading, setting up)15* device manager (flashing, upgrading, setting up)
16* Landing Manager (formerly “Trunk delivery system”)16* Landing Manager (formerly “Trunk delivery system”)
@@ -64,7 +64,7 @@
64 * Can deploy a prototype64 * Can deploy a prototype
65 * Build a package in PPA65 * Build a package in PPA
6666
67* Integration Test Runner67* Test Runner
6868
69 * Initial API defined for making test requests69 * Initial API defined for making test requests
70 * Can deploy a prototype service70 * Can deploy a prototype service
@@ -90,7 +90,7 @@
90 * Can initiate integration test90 * Can initiate integration test
91 * Can block trunk91 * Can block trunk
9292
93* Integration Test Runner93* Test Runner
9494
95 * Can perform an autopilot test to a touch device.95 * Can perform an autopilot test to a touch device.
9696
@@ -159,7 +159,7 @@
159159
160**January 23**160**January 23**
161161
162* Integration Test Runner (some or all of them are already being discussed in email by vila if i understand it correct)162* Test Runner (some or all of them are already being discussed in email by vila if i understand it correctly)
163163
164 * Test selection criteria164 * Test selection criteria
165 * Ticketing tracker interface165 * Ticketing tracker interface

Subscribers

People subscribed via source and target branches