Merge lp:~ralsina/tanuki-agent/fix-product-ref into lp:tanuki-agent

[Roberto Alsina]

Updated docs for test triggering scenarios and progress/results APIs

Updated docs for test triggering scenarios and progress/results APIs

[Bret Barker (noise)]

+1'ing with known FIXMEs remaining.

review: Approve

=== modified file 'docs/api-reference-products.md'
--- docs/api-reference-products.md	2016-01-04 20:06:45 +0000
+++ docs/api-reference-products.md	2016-01-04 20:33:35 +0000
@@ -4,8 +4,7 @@
 [TOC]
 
 
-Definitions
-============
+# Definitions
 
 -   **Manifest**: used to define a Product
     that is based on Snaps. This is optional for image-based workflows.
@@ -25,8 +24,7 @@
     tests against a target device.
 -   **Test Result**: metadata about output of a test run.
 
-Authentication
-============
+# Authentication
 
 All API endpoints are authenticated, which ensures private access
 to your product's information.
@@ -114,8 +112,7 @@
 login process) into a file called conf.ini. Note, this script is meant only as an example and not
 intended for production use.
 
-Errors
-============
+# Errors
 
 The APIs use standard HTTP response codes. Most errors will
 contain a human-readable "message" field in the JSON response body. Here
@@ -141,8 +138,7 @@
         <p>There was an temporary error on the server. The request may be retried after a short wait. If the problem persists please contact support. Scripts should use an exponential backoff to avoid hammering the servers with retries.
 </table>
 
-API Reference
-=============
+# API Reference
 
 Product Management
 ------------------
@@ -257,7 +253,7 @@
         "test-snap.foo"
       ],
       "release": "15.04-core",
-      "name": "test-product.e2e",
+      "name": "test-product.foo",
       "id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx",
       "created_at": "2016-01-04T17:34:39.093142"
     }
@@ -705,12 +701,6 @@
 has the following required fields:
 
 
--   ```image_name```: a unique name within an
-    organization that is combined with each
-    test case to produce the full set of test
-    opportunities. I.e. if you have two test cases with a
-    matching image_name, then two test opportunities will be created.
-    Maximum length is 200 characters.
 -   ```image_reference```: is opaque to the
     system; it is for your Provisioning Kit in your lab
     to interpret and use. It can be any JSON
@@ -727,51 +717,50 @@
     image\_reference is not a URL to a binary. Currently
     this ID is used for querying test results. Maximum length is 36
     characters.
--   ```upgrade\_snaps```: NOT IMPLEMENTED YET.
+-   ```channel```: **FIXME DESCRIBE**
+-   ```extra_snaps```: **FIXME DESCRIBE**
 
 
 <table class="table">
 <tr>
-    <th>Action  <th>Image Reference Creation
+    <th>Action  <th>Image Advertising
 <tr>
-    <td>URL     <td>https://spi.canonical.com/orgs/&lt;org-id&gt;/images
+    <td>URL     <td>https://spi.canonical.com/products/&lt;product-id&gt;/images
 <tr>
     <td>Method  <td>POST
 <tr>
     <td>Data    <td>
 <pre>
 {
-    image_reference: "http://foo.bar",
-    image_unique_id: some unique_id,
-    image_name: my_image,
-    upgrade_snaps: []
-}
-</pre>
-<tr>
-    <td>Response    <td><code>201 CREATED</code> with a list of test opportunity IDs
-<tr>
-    <td>Response    <td><code>404 NOT FOUND</code> if there is no test spec for this image_name
-<pre>
-{
-  "id": "resource-not-found",
-  "message": "Couldn't find a Spec for image name 'my_image'",
-  "url": null
-}
-</pre>
+  "image_unique_id": "DEADBEEF_83451039",
+  "extra_snaps": [],
+  "channel": "edge",
+  "image_reference": "http://foo.com/bar"
+}
+</pre>
+<tr>
+    <td>Response    <td><code>201 CREATED</code> with a list of test opportunity IDs and image unique IDs.
 <tr>
     <td>Example    <td>
 <pre>
-$ ./scripts/api_example.py -X POST conf.ini https://spi.canonical.com/orgs/docs-org/images --data \
-'{
-"image_reference": "http://foo.bar",
-"image_unique_id": "fooo",
-"image_name": "imgname",
-"upgrade_snaps": []
-}'
+$ ./scripts/api_example.py -X POST conf.ini https://spi.canonical.com/products/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/images --data \
+'
+{
+  "image_unique_id": "DEADBEEF_83451039",
+  "extra_snaps": [],
+  "channel": "edge",
+  "image_reference": "http://foo.com/bar"
+}
+'
 
 HTTP 201
 {
-  'ids': ['99787105-911f-4e23-bded-b0b8253aee11']
+  "ids": [
+    {
+      "image_unique_id": "DEADBEEF_83451039",
+      "test_opportunity_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
+    }
+  ]
 }
 </pre>
 </table>
@@ -785,43 +774,24 @@
 advertised to the system, however they can be re-advertised manually
 afterwards, re-triggering tests with their original context.
 
-The new revision advertised will be matched with all **Manifests** within the context **Organization**
+The new revision advertised will be matched with all **Products**
 referencing the given **Snap**, and **Test Opportunities** will be created for all **Test Specs** linked to those
-**Manifests**.
+**Products**.
 
 ![](images/image03.png)
 
 
-By modelling new / specific **Manifests** (Products) under the same
-**Organization** customers define the testing context they care about:
-
-
-
-1.  **Update from a Gold Master (GM) image**:
-
-    Product "base\_image\_reference" contains specific versions of
-    product-specific snaps reflecting one generation / portion of the
-    installed base. In this case, tests represent the act of updating
-    products of an specific generation to the advertised snap.
-
-
-2.  **Fresh Install (special case of above where GM == Ubuntu-Core)**:
-
-    Product "base\_image\_reference" does not include
-    product-specific snaps and tests represent, for instance, the act of
-    installing the advertised snap on top of a vanilla ubuntu-core
-    image.
-
-
-From the [Agent & Provisioning Kit](#agent-operation) point of view, testing contexts will
-be homogeneously defined via Test Opportunity base\_image\_reference and extra\_snaps.
+By modelling new / specific **Products** customers define the testing context they care about:
+
+
+**FIXME Describe the new world test triggering scenarios**
 
 Snap revision advertisement request requires the following
 fields:
 
 -   **name** The name of the snap package. Maximum length is 200 characters.
 -   **revision** The new snap's revision. An integer number.
-
+-   **channel** On which channel is the new snap revision advertised.
 
 
 Snap revision advertisement will return a list of test opportunity IDs and image unique IDs that can be used to [query for results](#querying-test-results) later.
@@ -830,42 +800,101 @@
 <tr>
     <th>Action  <th>Snap Revision Triggering
 <tr>
-    <td>URL     <td>https://spi.canonical.com/orgs/&lt;org-id&gt;/snap-revisions
-<tr>
-    <td>Method  <td>POST
-<tr>
-    <td>Data    <td>
-<pre>
-{
-    "name": name of snap package
-    "revision": revision number in the Store (integer)
-}
-</pre>
-<tr>
-    <td>Response    <td><code>201 CREATED</code> with a list of test opportunity IDs and image unique IDs
-<tr>
-    <td>Response    <td><code>404 NOT FOUND</code> if there is no matched Manifests
-<pre>
-{
-  "id": "resource-not-found",
-  "message": "",
-  "url": null
-}
-</pre>
-<tr>
-    <td>Example    <td>
-<pre>
-$ ./scripts/api_example.py -X POST conf.ini https://spi.canonical.com/orgs/docs-org/snap-revisions \
---data '
-> {
->   "name": "hello.world",
->   "revision": 123
-> }'
-
-HTTP 201
-{
-'ids': [{'test_opportunity_id': 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx',
-      	'image_unique_id': 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx'}]}
+    <td>URL     <td>https://spi.canonical.com/products/&lt;product-id&gt;/snaps
+<tr>
+    <td>Method  <td>POST
+<tr>
+    <td>Data    <td>
+<pre>
+{
+  "name": "test-snap.foo",
+  "revision": 2,
+  "channel": "edge"
+}
+</pre>
+<tr>
+    <td>Response    <td><code>201 CREATED</code> with a list of test opportunity IDs and image unique IDs
+<tr>
+    <td>Example    <td>
+<pre>
+$ ./scripts/api_example.py -X POST conf.ini https://spi.canonical.com/products/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/snaps \
+--data '
+{
+  "name": "test-snap.foo",
+  "revision": 2,
+  "channel": "edge"
+}'
+
+HTTP 201
+{
+  "ids": [
+    {
+      "image_unique_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
+      "test_opportunity_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
+    }
+  ]
+}
+</pre>
+</table>
+
+### Gold Master
+
+Gold Master advertisement requires the following fields:
+
+-  **snaps** A list of snaps that form the gold master image.
+-  **channel** On what channel is the gold master advertised.
+
+Gold master advertisement will return a list of test opportunity IDs and image unique IDs that can be used to [query for results](#querying-test-results) later.
+
+<table class="table">
+<tr>
+    <th>Action  <th>Gold Master Triggering
+<tr>
+    <td>URL     <td>https://spi.canonical.com/products/&lt;product-id&gt;/gold-masters
+<tr>
+    <td>Method  <td>POST
+<tr>
+    <td>Data    <td>
+<pre>
+{
+  "snaps": A list of snaps/revisions,
+  "channel": a channel name
+}
+</pre>
+<tr>
+    <td>Response    <td><code>201 CREATED</code> with a list of test opportunity IDs and image unique IDs
+<tr>
+    <td>Example    <td>
+<pre>
+$ ./scripts/api_example.py -X POST conf.ini https://spi.canonical.com/products/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/snaps \
+--data '
+{
+  "snaps": [
+    {
+      "name": "pi2.canonical",
+      "revision": 1
+    },
+    {
+      "name": "webdm.canonical",
+      "revision": 1
+    },
+    {
+      "name": "test-snap.foo",
+      "revision": 1
+    }
+  ],
+  "channel": "stable"
+}'
+
+HTTP 201
+{
+  "ids": [
+    {
+      "image_unique_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
+      "test_opportunity_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
+    }
+  ]
+}
 </pre>
 </table>
 
@@ -873,8 +902,6 @@
 Querying Test Progress
 ----------------------
 
-
-
 As a test opportunity progresses from being triggered to deliver a
 result it flows through a series of expected events. This is the normal
 expected event flow for a test opportunity:
@@ -898,7 +925,7 @@
 <tr>
     <th>Action  <th>Result Query
 <tr>
-    <td>URL     <td>https://spi.canonical.com/orgs/&lt;org_id&gt;/tests/events
+    <td>URL     <td>https://spi.canonical.com/products/&lt;prod_id&gt;/tests/events
 <tr>
     <td>Method  <td>GET
 <tr>
@@ -926,54 +953,43 @@
     <td>Example    <td>
 <pre>
 $ ./scripts/api_example.py conf.ini \
-http://spi.canonical.com/orgs/e2e-org/tests/events?test_opportunity_id=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
+http://spi.canonical.com/products/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/tests/events?test_opportunity_id=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
 HTTP 200
 {
   "event_logs": [
     {
-      "organization_id": "e2e-org",
-      "image_name": "e2e-imgf151a8cd9081404db54ed1d8dab1e140",
-      "queued_at": "2015-09-17T12:13:03.288000+00:00",
-      "updated_at": "2015-09-17T12:13:06.799000+00:00",
-      "image_unique_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
-      "events_seen": [
-        "QUEUED_FOR_AGENTS",
-        "PICKED_BY_AGENT",
-        "RESULT_POSTED",
-        "RESULT_STORED"
-      ],
       "events": [
         {
           "event_type": "QUEUED_FOR_AGENTS",
-          "timestamp": "2015-09-17T12:13:03.288000+00:00"
+          "timestamp": "2016-01-04T16:58:28.805000+00:00"
         },
         {
+          "timestamp": "2016-01-04T16:58:33.185000+00:00",
           "event_type": "PICKED_BY_AGENT",
-          "agent_group": "e2e-group",
-          "agent_id": "e2e-id",
-          "timestamp": "2015-09-17T12:13:06.291000+00:00",
-          "platform": "amd64"
+          "agent_id": "foo-agent-id",
+          "platform": null,
+          "agent_group": null
         },
         {
           "result_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
           "test_status": "PASSED",
-          "agent_group": "e2e-group",
           "event_type": "RESULT_POSTED",
-          "timestamp": "2015-09-17T12:13:06.734000+00:00",
-          "agent_id": "e2e-id",
-          "platform": "amd64"
+          "agent_id": "foo-agent-id",
+          "platform": null,
+          "agent_group": null,
+          "timestamp": "2016-01-04T16:58:33.734000+00:00"
         },
         {
+          "result_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
+          "test_status": "PASSED",
           "event_type": "RESULT_STORED",
-          "result_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
-          "test_status": "PASSED",
-          "timestamp": "2015-09-17T12:13:06.745000+00:00"
+          "timestamp": "2016-01-04T16:58:33.764000+00:00"
         }
       ],
+      "updated_at": "2016-01-04T16:58:33.825000+00:00",
+      "product_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
+      "queued_at": "2016-01-04T16:58:28.805000+00:00",
       "test_opportunity_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
-      "spec_name": "e2e-spec_13691451",
-      "test_opportunity": {
-
 (...)
 </pre>
 </table>
@@ -1028,7 +1044,7 @@
 <tr>
     <th>Action  <th>Result Query
 <tr>
-    <td>URL     <td>https://spi.canonical.com/orgs/&lt;org_id&gt;/results
+    <td>URL     <td>https://spi.canonical.com/products/&lt;productd_id&gt;/tests/results
 <tr>
     <td>Method  <td>GET
 <tr>
@@ -1049,30 +1065,19 @@
     <td>Example    <td>
 <pre>
 ./scripts/api_example.py conf.ini \
-'https://spi.canonical.com/orgs/docs-org/results?start_date=20150826T00%3A00%3A00&end_date=20150826T04%3A00%3A00'
+'https://spi.canonical.com/products/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/results?start_date=20150826T00%3A00%3A00&end_date=20150826T04%3A00%3A00'
 
 HTTP 200
 {
   "test_results": [
     {
-      "test_spec": {
-        "name": "e2e-spec_70600150",
-        "platform": "amd64",
-        "image_name": "e2e-img",
-        "id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
-        "created_at": "2015-08-26T00:00:20.297477",
-        "test_payload": "{'foo': 'bar'}"
-      },
       "test_status": "PASSED",
-      "platform": "amd64",
-      "organization_id": "docs-org",
-      "image_name": "e2e-img",
-      "extra_snaps": [],
-      "image_reference": "http://foo.com/bar",
-      "agent": {
-        "timing": {
-          "setup_start": "2015-08-26T00:00:22.651078",
-
+      "agent_revno": "197",
+      "image_reference": null,
+      "test_opportunity_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
+      "primary_snap_name": "test-snap.foo",
+      "base_channel": "stable",
+      "update_channel": null,
 [...]
 </pre>
 </table>
@@ -1093,12 +1098,8 @@
 -   20150914 17:51:31
 -   20150914 175131 Z
 
-
-
 ### Test Result Data
 
-
-
 There are a few key fields in the test result JSON of note:
 
 <!-- FIX link to actors-and-results -->
@@ -1141,7 +1142,7 @@
     {
       "image_unique_id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
       "test_status": "PASSED",
-      "agent-id": "e2e-id",
+      "agent-id": "foo-agent-id",
       "id": "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
       "result_payload": {
         "foo": "bar"