Linaro Dashboard Bundle utilities

Merge lp:~stylesen/linaro-python-dashboard-bundle/evolve-bundle-stream-1.6 into lp:linaro-python-dashboard-bundle/linaro-11.11

  1. evolve-bundle-stream-1.6
  2. Merge into trunk
Proposed by Senthil Kumaran S
Status: Merged
Merged at revision: 75
Proposed branch: lp:~stylesen/linaro-python-dashboard-bundle/evolve-bundle-stream-1.6
Merge into: lp:linaro-python-dashboard-bundle/linaro-11.11
Diff against target: 675 lines (+600/-2)
7 files modified
linaro_dashboard_bundle/evolution.py (+12/-0)
linaro_dashboard_bundle/io.py (+5/-0)
linaro_dashboard_bundle/schemas/dashboard_bundle_format_1.6.json (+345/-0)
linaro_dashboard_bundle/test_documents/everything_in_one_bundle_1.6.json (+102/-0)
linaro_dashboard_bundle/test_documents/evolution_1.6.json (+70/-0)
linaro_dashboard_bundle/test_documents/example_1.6_test_result_testdef_metadata.json (+28/-0)
linaro_dashboard_bundle/tests.py (+38/-2)
To merge this branch: bzr merge lp:~stylesen/linaro-python-dashboard-bundle/evolve-bundle-stream-1.6
Reviewer Review Type Date Requested Status
Antonio Terceiro Approve
Zygmunt Krynicki (community) Needs Fixing
Linaro Validation Team Pending
Linaro Validation Team Pending
Linaro Validation Team Pending
Review via email: mp+150520@code.launchpad.net

Description of the change

Evolve bundle stream version to 1.6 due to the addition of testdef metadata.

To post a comment you must log in.
Revision history for this message
Zygmunt Krynicki (zyga) wrote :

17 + testdef_metadata = {
18 + "version": "5",
19 + "description": "sample test definition",
20 + "format": "Lava Test Shell Format 1.0",
21 + "location": "LOCAL",
22 + "url": "http://localhost/",
23 + "environment": "lava-test-shell",
24 + "os": "ubuntu",
25 + "devices": "panda"
26 + }

Why are you adding that to _all_ documents that previously had no metadata? Do you assume that everything is running ubuntu on a panda board?

-1

review: Disapprove
Revision history for this message
Senthil Kumaran S (stylesen) wrote :

Hi Zyga,

On Tuesday 26 February 2013 04:14 PM, Zygmunt Krynicki wrote:
> Review: Disapprove
>
> 17 + testdef_metadata = {
> 18 + "version": "5",
> 19 + "description": "sample test definition",
> 20 + "format": "Lava Test Shell Format 1.0",
> 21 + "location": "LOCAL",
> 22 + "url": "http://localhost/",
> 23 + "environment": "lava-test-shell",
> 24 + "os": "ubuntu",
> 25 + "devices": "panda"
> 26 + }
>
> Why are you adding that to _all_ documents that previously had no metadata? Do you assume that everything is running ubuntu on a panda board?

My understanding of the above code is when we want to evolve from 1.5 to
1.6 we need to add the above to 1.5 format which will fetch us 1.6
format. This was required by one of the test cases in the test suite for
linaro-python-dashboard-bundle, hence I added it.

Let me know the correct way of doing it.

Thank You.
--
Senthil Kumaran S
http://www.stylesen.org/
http://www.sasenthilkumaran.com/

Revision history for this message
Zygmunt Krynicki (zyga) wrote :

> Hi Zyga,
>
> My understanding of the above code is when we want to evolve from 1.5 to
> 1.6 we need to add the above to 1.5 format which will fetch us 1.6
> format. This was required by one of the test cases in the test suite for
> linaro-python-dashboard-bundle, hence I added it.

You are missing the point. You may have silenced some tests but the code just makes no sense. If someone goes and imports an older run from lava-test the patch that I commented on will add plainly broken meta-data.

I suspect you want an empty default meta data instead. Only tests that actually use the new test definitions should have that section filled in. This is similar to what most recent evolutions were doing, namely, adding optional values that have empty defaults.

Thanks
ZK

lp:~stylesen/linaro-python-dashboard-bundle/evolve-bundle-stream-1.6 updated
76. By Senthil Kumaran S

Use empty testdef_metadata when we evolve from 1.5 to 1.6

Revision history for this message
Senthil Kumaran S (stylesen) wrote :

Hi Zyga,

On Tuesday 26 February 2013 04:37 PM, Zygmunt Krynicki wrote:
> I suspect you want an empty default meta data instead. Only tests that actually use the new test definitions should have that section filled in. This is similar to what most recent evolutions were doing, namely, adding optional values that have empty defaults.

I just committed a patch and re-requested review.

Thank You.
--
Senthil Kumaran S
http://www.stylesen.org/
http://www.sasenthilkumaran.com/

Revision history for this message
Zygmunt Krynicki (zyga) wrote :

Hi

There's still one subtle issue with this patch:

17 + testdef_metadata = {
18 + "version": "",
19 + "description": "",
20 + "format": "",
21 + "location": "",
22 + "url": "",
23 + "environment": "",
24 + "os": "",
25 + "devices": ""
26 + }

This adds values where there previously were none. That is, a 1.5 record without meta-data will look different to a 1.6 record without meta data. The migrated 1.5 version will have empty strings everywhere. The 1.6 record will just have no values there (allowing defaults to be used instead).

Looking a the schema now:

272 + "testdef_metadata": {
273 + "description": "Container for test definition metadata defined by the test and their values during this particular run",
274 + "type": "object",
275 + "optional": true,
276 + "additionalProperties": false,
277 + "properties": {
278 + "description": {
279 + "description": "Description about the test.",
280 + "type": "string",
281 + "optional": true
282 + },
283 + "version": {
284 + "description": "Version of test definition if any.",
285 + "type": "string",
286 + "optional": true
287 + },
288 + "format": {
289 + "description": "Format of test definition.",
290 + "type": "string",
291 + "optional": true
292 + },
293 + "location": {
294 + "description": "Location of test definition.",
295 + "type": "string",
296 + "optional": true
297 + },
298 + "url": {
299 + "description": "URL of test definition.",
300 + "type": "string",
301 + "optional": true
302 + },
303 + "environment": {
304 + "description": "Test definition environment.",
305 + "type": "string",
306 + "optional": true
307 + },
308 + "os": {
309 + "description": "Test definition OSs.",
310 + "type": "string",
311 + "optional": true
312 + },
313 + "devices": {
314 + "description": "Test definition devices.",
315 + "type": "string",
316 + "optional": true
317 + }
318 + }
319 + },

I see that everything is optional and nothing has a default value.

I don't know the problem you are currently solving but my suggestion would be to use default: None everywhere (except for the structure where default should be {}) and actually use those defaults when referring to particular elements.

This can be simplified with json-document (which does not need to be a part of this repository) or by manually using code like:

value = container.get(attr, schema[attr]["default"])

So I'm still -1 on that unless mwhudson feels otherwise

review: Needs Fixing
Revision history for this message
Zygmunt Krynicki (zyga) wrote :

Any progress on this one?

review: Needs Information
Revision history for this message
Senthil Kumaran S (stylesen) wrote :

Hi Zyga,

On Monday 04 March 2013 03:50 PM, Zygmunt Krynicki wrote:
> Any progress on this one?

Will give an update shortly.

Thank You.
--
Senthil Kumaran S
http://www.stylesen.org/
http://www.sasenthilkumaran.com/

Revision history for this message
Michael Hudson-Doyle (mwhudson) wrote :

Zygmunt Krynicki <email address hidden> writes:

> So I'm still -1 on that unless mwhudson feels otherwise

mwhudson agrees with you.

lp:~stylesen/linaro-python-dashboard-bundle/evolve-bundle-stream-1.6 updated
77. By Senthil Kumaran S

Provide default values for testdef_metadata parameters in the schema and fix
the tests accordingly.

Revision history for this message
Senthil Kumaran S (stylesen) wrote :

Hi zyga/mwhudson,

Apologies for taking a long time on this review. I just made a patch which fixes the issues raised by zyga on the previous comment. I am in the process of intergrating it along with the other changes to lava-dashboard, where mwhudson has given many review comments. I shall do that too and re-request for a review once done.

Thank You.

Revision history for this message
Senthil Kumaran S (stylesen) wrote :

I just re-requested a review for this branch.

Revision history for this message
Zygmunt Krynicki (zyga) wrote :

52 - "testdef_metadata": {
53 - "description": "Container for test definition metadata defined by the test and their values during this particular run",
54 - "type": "object",
55 - "optional": true,
56 - "additionalProperties": {
57 - "description": "Properties that are defined in the test definiton",
58 - "type": "string"
59 - }
60 - },

This is wrong, you cannot change existing schema

647 - def test_evoloved_document_is_what_we_expect(self):
648 + def test_evolved_document_is_what_we_expect(self):

Propose fixes for typos separately

review: Needs Fixing
Revision history for this message
Senthil Kumaran S (stylesen) wrote :

On Tue, 2013-03-26 at 14:02 +0000, Zygmunt Krynicki wrote:
> 52 - "testdef_metadata": {
> 53 - "description": "Container for test definition metadata defined by the test and their values during this particular run",
> 54 - "type": "object",
> 55 - "optional": true,
> 56 - "additionalProperties": {
> 57 - "description": "Properties that are defined in the test definiton",
> 58 - "type": "string"
> 59 - }
> 60 - },
>
> This is wrong, you cannot change existing schema
>

This is part of the changes included in this branch. I just introduced
it part of the work am doing for evolve-bundle-stream-1.6

> 647 - def test_evoloved_document_is_what_we_expect(self):
> 648 + def test_evolved_document_is_what_we_expect(self):
>
> Propose fixes for typos separately

Ok I shall revert this change.

Thank You.
--
Senthil Kumaran
http://www.stylesen.org/
http://www.sasenthilkumaran.com/

lp:~stylesen/linaro-python-dashboard-bundle/evolve-bundle-stream-1.6 updated
78. By Senthil Kumaran S

Revert typo fixes which has to be requested separately.

Revision history for this message
Senthil Kumaran S (stylesen) wrote :

> > 647 - def test_evoloved_document_is_what_we_expect(self):
> > 648 + def test_evolved_document_is_what_we_expect(self):
> >
> > Propose fixes for typos separately
>
> Ok I shall revert this change.

I reverted and pushed this change.

lp:~stylesen/linaro-python-dashboard-bundle/evolve-bundle-stream-1.6 updated
79. By Senthil Kumaran S

Revert the changes to bundle format 1.5 which shouldn't be touched once it
is accepted and pushed to the repository.

Revision history for this message
Senthil Kumaran S (stylesen) wrote :

On Tue, 2013-03-26 at 14:02 +0000, Zygmunt Krynicki wrote:
> Review: Needs Fixing
>
> 52 - "testdef_metadata": {
> 53 - "description": "Container for test definition metadata defined by the test and their values during this particular run",
> 54 - "type": "object",
> 55 - "optional": true,
> 56 - "additionalProperties": {
> 57 - "description": "Properties that are defined in the test definiton",
> 58 - "type": "string"
> 59 - }
> 60 - },

This was pushed in last release. I got your point. I reverted this
change too. I shall re-request review.

Thank You.
--
Senthil Kumaran
http://www.stylesen.org/
http://www.sasenthilkumaran.com/

Revision history for this message
Antonio Terceiro (terceiro) wrote :

Looks good to me now.

Also, Zygmunt's and Michael's concerns are solved in the current state of the branch.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'linaro_dashboard_bundle/evolution.py'
2--- linaro_dashboard_bundle/evolution.py 2012-11-06 21:08:45 +0000
3+++ linaro_dashboard_bundle/evolution.py 2013-03-26 15:05:28 +0000
4@@ -169,6 +169,15 @@
5 assert doc.get("format") == "Dashboard Bundle Format 1.4"
6 doc["format"] = "Dashboard Bundle Format 1.5"
7
8+ def _evolution_from_1_5_to_1_6(doc):
9+ """
10+ Evolution method for 1.5 -> 1.6:
11+
12+ * No changes required
13+ """
14+ assert doc.get("format") == "Dashboard Bundle Format 1.5"
15+ doc["format"] = "Dashboard Bundle Format 1.6"
16+
17 EVOLUTION_PATH = [
18 ("Dashboard Bundle Format 1.0",
19 "Dashboard Bundle Format 1.0.1",
20@@ -188,4 +197,7 @@
21 ("Dashboard Bundle Format 1.4",
22 "Dashboard Bundle Format 1.5",
23 _evolution_from_1_4_to_1_5),
24+ ("Dashboard Bundle Format 1.5",
25+ "Dashboard Bundle Format 1.6",
26+ _evolution_from_1_5_to_1_6),
27 ]
28
29=== modified file 'linaro_dashboard_bundle/io.py'
30--- linaro_dashboard_bundle/io.py 2012-11-06 20:57:42 +0000
31+++ linaro_dashboard_bundle/io.py 2013-03-26 15:05:28 +0000
32@@ -71,6 +71,11 @@
33 resource_string(
34 __name__,
35 'schemas/dashboard_bundle_format_1.5.json'))),
36+ 'Dashboard Bundle Format 1.6': Schema(
37+ json.loads(
38+ resource_string(
39+ __name__,
40+ 'schemas/dashboard_bundle_format_1.6.json'))),
41 }
42
43 @classmethod
44
45=== added file 'linaro_dashboard_bundle/schemas/dashboard_bundle_format_1.6.json'
46--- linaro_dashboard_bundle/schemas/dashboard_bundle_format_1.6.json 1970-01-01 00:00:00 +0000
47+++ linaro_dashboard_bundle/schemas/dashboard_bundle_format_1.6.json 2013-03-26 15:05:28 +0000
48@@ -0,0 +1,345 @@
49+{
50+ "description": "DashboardBundle object",
51+ "type": "object",
52+ "additionalProperties": false,
53+ "properties": {
54+ "format": {
55+ "description": "Document format identifier.",
56+ "type": "string",
57+ "enum": [
58+ "Dashboard Bundle Format 1.6"
59+ ]
60+ },
61+ "test_runs": {
62+ "description": "Array of TestRun objects",
63+ "type": "array",
64+ "optional": true,
65+ "items": {
66+ "description": "TestRun object",
67+ "type": "object",
68+ "additionalProperties": false,
69+ "properties": {
70+ "analyzer_assigned_uuid": {
71+ "description": "UUID that was assigned by the log analyzer during processing",
72+ "type": "string"
73+ },
74+ "analyzer_assigned_date": {
75+ "description": "Time stamp in ISO 8601 format that was assigned by the log analyzer during processing. The exact format is YYYY-MM-DDThh:mm:ssZ",
76+ "type": "string",
77+ "format": "date-time"
78+ },
79+ "time_check_performed": {
80+ "description": "Indicator on whether the log analyzer had accurate time information",
81+ "type": "boolean"
82+ },
83+ "test_duration": {
84+ "description": "Duration of the test run. Duration is stored in the following format '[DAYS]d [SECONDS]s [MICROSECONDS]us'",
85+ "type": "string",
86+ "optional": true
87+ },
88+ "attributes": {
89+ "description": "Container for additional attributes defined by the test and their values during this particular run",
90+ "type": "object",
91+ "optional": true,
92+ "additionalProperties": {
93+ "description": "Arbitrary properties that are defined by the test",
94+ "type": "string"
95+ }
96+ },
97+ "tags": {
98+ "description": "An optional array of tags that are associated with this test run. Note that those are not version-control tags but arbitrary classifiers for test results",
99+ "type": "array",
100+ "optional": true,
101+ "items": {
102+ "description": "Tag name",
103+ "type": "string"
104+ }
105+ },
106+ "test_id": {
107+ "description": "Test identifier. Must be a well-defined (in scope of the dashboard) name of the test",
108+ "type": "string"
109+ },
110+ "test_results": {
111+ "description": "Array of TestResult objects",
112+ "type": "array",
113+ "items": {
114+ "description": "TestResult object",
115+ "type": "object",
116+ "additionalProperties": false,
117+ "properties": {
118+ "test_case_id": {
119+ "description": "Identifier of the TestCase this test result came from",
120+ "type": "string",
121+ "optional": true
122+ },
123+ "result": {
124+ "description": "Status code of this test result",
125+ "type": "string",
126+ "enum": ["pass", "fail", "skip", "unknown"]
127+ },
128+ "message": {
129+ "description": "Message scrubbed from the log file",
130+ "type": "string",
131+ "optional": true
132+ },
133+ "measurement": {
134+ "description": "Numerical measurement associated with the test result",
135+ "type": "number",
136+ "optional": true,
137+ "requires": "test_case_id"
138+ },
139+ "units": {
140+ "description": "Units for measurement",
141+ "type": "string",
142+ "optional": true,
143+ "requires": "measurement"
144+ },
145+ "timestamp": {
146+ "description": "Date and time when the test was performed",
147+ "type": "string",
148+ "optional": true,
149+ "format": "date-time"
150+ },
151+ "duration": {
152+ "description": "Duration of the test case. Duration is stored in the following format '[DAYS]d [SECONDS]s [MICROSECONDS]us'",
153+ "type": "string",
154+ "optional": true
155+ },
156+ "log_filename": {
157+ "description": "Filename of the log file which this test result was scrubbed from",
158+ "type": "string",
159+ "optional": true
160+ },
161+ "log_lineno": {
162+ "description": "Precise location in the log file (line number)",
163+ "type": "integer",
164+ "optional": true,
165+ "requires": "log_filename"
166+ },
167+ "attributes": {
168+ "description": "Container for additional attributes defined by test result",
169+ "type": "object",
170+ "optional": true,
171+ "additionalProperties": {
172+ "description": "Arbitrary properties that are defined by the particular test result",
173+ "type": "string"
174+ }
175+ },
176+ "attachments": {
177+ "description": "Array of attachments",
178+ "optional": true,
179+ "type": "array",
180+ "items": {
181+ "type": "object",
182+ "additionalProperties": false,
183+ "properties": {
184+ "pathname": {
185+ "description": "Attachment pathname",
186+ "type": "string"
187+ },
188+ "mime_type": {
189+ "description": "Attachment MIME type",
190+ "type": "string"
191+ },
192+ "content": {
193+ "description": "Attachment content encoded as base64 string with standard encoding",
194+ "type": "string",
195+ "optional": true
196+ },
197+ "public_url": {
198+ "description": "Public URL of this attachment",
199+ "type": "string",
200+ "optional": true
201+ }
202+ }
203+ }
204+ }
205+ }
206+ }
207+ },
208+ "attachments": {
209+ "description": "Array of attachments",
210+ "optional": true,
211+ "type": "array",
212+ "items": {
213+ "type": "object",
214+ "additionalProperties": false,
215+ "properties": {
216+ "pathname": {
217+ "description": "Attachment pathname",
218+ "type": "string"
219+ },
220+ "mime_type": {
221+ "description": "Attachment MIME type",
222+ "type": "string"
223+ },
224+ "content": {
225+ "description": "Attachment content encoded as base64 string with standard encoding",
226+ "type": "string",
227+ "optional": true
228+ },
229+ "public_url": {
230+ "description": "Public URL of this attachment",
231+ "type": "string",
232+ "optional": true
233+ }
234+ }
235+ }
236+ },
237+ "testdef_metadata": {
238+ "description": "Container for test definition metadata defined by the test and their values during this particular run",
239+ "type": "object",
240+ "optional": true,
241+ "additionalProperties": false,
242+ "properties": {
243+ "description": {
244+ "description": "Description about the test.",
245+ "type": "string",
246+ "default": null
247+ },
248+ "version": {
249+ "description": "Version of test definition if any.",
250+ "type": "string",
251+ "default": null
252+ },
253+ "format": {
254+ "description": "Format of test definition.",
255+ "type": "string",
256+ "default": null
257+ },
258+ "location": {
259+ "description": "Location of test definition.",
260+ "type": "string",
261+ "default": null
262+ },
263+ "url": {
264+ "description": "URL of test definition.",
265+ "type": "string",
266+ "default": null
267+ },
268+ "environment": {
269+ "description": "Test definition environment.",
270+ "type": "string",
271+ "default": null
272+ },
273+ "os": {
274+ "description": "Test definition OSs.",
275+ "type": "string",
276+ "default": null
277+ },
278+ "devices": {
279+ "description": "Test definition devices.",
280+ "type": "string",
281+ "default": null
282+ }
283+ }
284+ },
285+ "hardware_context": {
286+ "description": "Description of the hardware context in which this test was running",
287+ "type": "object",
288+ "optional": true,
289+ "additionalProperties": false,
290+ "properties": {
291+ "devices": {
292+ "description": "Array of HardwareDevice objects",
293+ "type": "array",
294+ "items": {
295+ "description": "HardwareDevice object",
296+ "type": "object",
297+ "properties": {
298+ "device_type": {
299+ "type": "string",
300+ "description": "Device type"
301+ },
302+ "description": {
303+ "type": "string",
304+ "description": "Human readable description of the device"
305+ },
306+ "attributes": {
307+ "description": "Container for additional attributes defined by the device",
308+ "type": "object",
309+ "optional": true,
310+ "additionalProperties": {
311+ "description": "Arbitrary properties that are defined by the particular hardware device",
312+ "type": ["number", "string"]
313+ }
314+ }
315+ },
316+ "additionalProperties": false
317+ }
318+ }
319+ }
320+ },
321+ "software_context": {
322+ "description": "Description of the software context in which this test was running",
323+ "type": "object",
324+ "additionalProperties": false,
325+ "optional": true,
326+ "properties": {
327+ "image": {
328+ "description": "SoftwareImage object",
329+ "type": "object",
330+ "optional": true,
331+ "additionalProperties": false,
332+ "properties": {
333+ "name": {
334+ "description": "Name of the operating system image",
335+ "type": "string"
336+ }
337+ }
338+ },
339+ "sources": {
340+ "type": "array",
341+ "optional": true,
342+ "items": {
343+ "type": "object",
344+ "additionalProperties": false,
345+ "properties": {
346+ "project_name": {
347+ "type": "string"
348+ },
349+ "branch_vcs": {
350+ "type": "string",
351+ "enum": ["bzr", "git", "svn"]
352+ },
353+ "branch_url": {
354+ "type": "string"
355+ },
356+ "branch_revision": {
357+ "type": ["string", "integer"]
358+ },
359+ "commit_timestamp": {
360+ "type": "string",
361+ "format": "date-time",
362+ "optional": true
363+ }
364+ }
365+ }
366+ },
367+ "packages": {
368+ "description": "Array of SoftwarePackage objects",
369+ "type": "array",
370+ "optional": true,
371+ "items": {
372+ "description": "SoftwarePackage object",
373+ "type": "object",
374+ "additionalProperties": false,
375+ "properties": {
376+ "name": {
377+ "description": "Package name",
378+ "type": "string"
379+ },
380+ "version": {
381+ "description": "Package version",
382+ "type": "string"
383+ }
384+ }
385+ }
386+ }
387+ }
388+ }
389+ }
390+ }
391+ }
392+ }
393+}
394
395=== added file 'linaro_dashboard_bundle/test_documents/everything_in_one_bundle_1.6.json'
396--- linaro_dashboard_bundle/test_documents/everything_in_one_bundle_1.6.json 1970-01-01 00:00:00 +0000
397+++ linaro_dashboard_bundle/test_documents/everything_in_one_bundle_1.6.json 2013-03-26 15:05:28 +0000
398@@ -0,0 +1,102 @@
399+{
400+ "format": "Dashboard Bundle Format 1.6",
401+ "test_runs": [
402+ {
403+ "analyzer_assigned_date": "2010-11-14T13:42:31Z",
404+ "analyzer_assigned_uuid": "01234567-0123-0123-0123-01234567890A",
405+ "test_id": "example test id",
406+ "time_check_performed": false,
407+ "test_duration": "0d 1s 134us",
408+ "attributes": {
409+ "attr1": "value1",
410+ "attr2": "value2"
411+ },
412+ "tags": [
413+ "example-content",
414+ "example-content-format-1.3"
415+ ],
416+ "attachments": [
417+ {
418+ "pathname": "attachment1.txt",
419+ "mime_type": "text/plain",
420+ "content": "bGluZTEKbGluZTIKbGluZTMK"
421+ },
422+ {
423+ "pathname": "attachment2.txt",
424+ "mime_type": "text/plain",
425+ "content": "b3RoZXIgbGluZTEK",
426+ "public_url": "http://www.example.org/attachment2.txt"
427+ }
428+ ],
429+ "testdef_metadata": {
430+ "version": "5",
431+ "description": "sample test definition",
432+ "format": "Lava Test Shell Format 1.0",
433+ "location": "LOCAL",
434+ "url": "http://localhost/",
435+ "environment": "lava-test-shell",
436+ "os": "ubuntu",
437+ "devices": "panda"
438+ },
439+ "test_results": [
440+ {
441+ "test_case_id": "example test case id",
442+ "result": "pass",
443+ "message": "example message",
444+ "measurement": 3.5,
445+ "units": "s",
446+ "timestamp": "2010-11-14T13:49:56Z",
447+ "duration": "0d 1s 134us",
448+ "log_filename": "attachment1.txt",
449+ "log_lineno": 2,
450+ "attributes": {
451+ "test result attribute": "value"
452+ },
453+ "attachments": [
454+ {
455+ "pathname": "attachment1.txt",
456+ "mime_type": "text/plain",
457+ "content": "bGluZTEKbGluZTIKbGluZTMK"
458+ }
459+ ]
460+ }
461+ ],
462+ "hardware_context": {
463+ "devices": [
464+ {
465+ "device_type": "example device type",
466+ "description": "example device",
467+ "attributes": {
468+ "hw attr1": "value1",
469+ "hw attr2": "value2"
470+ }
471+ }
472+ ]
473+ },
474+ "software_context": {
475+ "packages": [
476+ {
477+ "name": "pkg1",
478+ "version": "version1"
479+ },
480+ {
481+ "name": "pkg2",
482+ "version": "version2"
483+ }
484+ ],
485+ "image": {
486+ "name": "example os image"
487+ },
488+ "sources": [
489+ {
490+ "branch_revision": 93556,
491+ "branch_url": "lp:gcc-linaro/4.4",
492+ "branch_vcs": "bzr",
493+ "commit_timestamp": "2010-09-07T14:49:43Z",
494+ "project_name": "linaro-gcc"
495+ }
496+ ]
497+ }
498+ }
499+ ]
500+}
501
502=== added file 'linaro_dashboard_bundle/test_documents/evolution_1.6.json'
503--- linaro_dashboard_bundle/test_documents/evolution_1.6.json 1970-01-01 00:00:00 +0000
504+++ linaro_dashboard_bundle/test_documents/evolution_1.6.json 2013-03-26 15:05:28 +0000
505@@ -0,0 +1,70 @@
506+{
507+ "format": "Dashboard Bundle Format 1.6",
508+ "test_runs": [
509+ {
510+ "analyzer_assigned_date": "2010-11-14T13:42:31Z",
511+ "analyzer_assigned_uuid": "01234567-0123-0123-0123-01234567890A",
512+ "test_id": "example test id",
513+ "time_check_performed": false,
514+ "attributes": {
515+ "attr1": "value1",
516+ "attr2": "value2"
517+ },
518+ "attachments": [
519+ {
520+ "pathname": "attachment1.txt",
521+ "mime_type": "text/plain",
522+ "content": "bGluZTEKbGluZTIKbGluZTMK"
523+ },
524+ {
525+ "pathname": "attachment2.txt",
526+ "mime_type": "text/plain",
527+ "content": "b3RoZXIgbGluZTEK"
528+ }
529+ ],
530+ "test_results": [
531+ {
532+ "test_case_id": "example test case id",
533+ "result": "pass",
534+ "message": "example message",
535+ "measurement": 3.5,
536+ "units": "s",
537+ "timestamp": "2010-11-14T13:49:56Z",
538+ "duration": "0d 1s 134us",
539+ "log_filename": "attachment1.txt",
540+ "log_lineno": 2,
541+ "attributes": {
542+ "test result attribute": "value"
543+ }
544+ }
545+ ],
546+ "hardware_context": {
547+ "devices": [
548+ {
549+ "device_type": "example device type",
550+ "description": "example device",
551+ "attributes": {
552+ "hw attr1": "value1",
553+ "hw attr2": "value2"
554+ }
555+ }
556+ ]
557+ },
558+ "software_context": {
559+ "packages": [
560+ {
561+ "name": "pkg1",
562+ "version": "version1"
563+ },
564+ {
565+ "name": "pkg2",
566+ "version": "version2"
567+ }
568+ ],
569+ "image": {
570+ "name": "example os image"
571+ }
572+ }
573+ }
574+ ]
575+}
576
577=== added file 'linaro_dashboard_bundle/test_documents/example_1.6_test_result_testdef_metadata.json'
578--- linaro_dashboard_bundle/test_documents/example_1.6_test_result_testdef_metadata.json 1970-01-01 00:00:00 +0000
579+++ linaro_dashboard_bundle/test_documents/example_1.6_test_result_testdef_metadata.json 2013-03-26 15:05:28 +0000
580@@ -0,0 +1,28 @@
581+{
582+ "format": "Dashboard Bundle Format 1.6",
583+ "test_runs": [
584+ {
585+ "analyzer_assigned_date": "2010-11-14T13:42:31Z",
586+ "analyzer_assigned_uuid": "01234567-0123-0123-0123-01234567890A",
587+ "test_id": "example test id",
588+ "time_check_performed": false,
589+ "test_results": [
590+ {
591+ "test_case_id": "example test case id",
592+ "result": "pass",
593+ "attachments": []
594+ },
595+ ],
596+ "testdef_metadata": {
597+ "version": "5",
598+ "description": "sample test definition",
599+ "format": "Lava Test Shell Format 1.0",
600+ "location": "LOCAL",
601+ "url": "http://localhost/",
602+ "environment": "lava-test-shell",
603+ "os": "ubuntu",
604+ "devices": "panda"
605+ }
606+ }
607+ ]
608+}
609
610=== modified file 'linaro_dashboard_bundle/tests.py'
611--- linaro_dashboard_bundle/tests.py 2012-11-06 21:09:21 +0000
612+++ linaro_dashboard_bundle/tests.py 2013-03-26 15:05:28 +0000
613@@ -228,6 +228,9 @@
614 ('everything_in_one_bundle_1_5', {
615 'filename': 'everything_in_one_bundle_1.5.json'
616 }),
617+ ('everything_in_one_bundle_1_6', {
618+ 'filename': 'everything_in_one_bundle_1.6.json'
619+ }),
620 ]
621
622 def test_load_document(self):
623@@ -458,17 +461,50 @@
624 def test_evolved_document_is_latest_format(self):
625 self.assertFalse(DocumentEvolution.is_latest(self.doc))
626 DocumentEvolution.evolve_document(self.doc, one_step=True)
627+ self.assertFalse(DocumentEvolution.is_latest(self.doc))
628+
629+ def test_evolved_document_is_valid(self):
630+ DocumentEvolution.evolve_document(self.doc, one_step=True)
631+ self.assertEqual(DocumentIO.check(self.doc),
632+ "Dashboard Bundle Format 1.5")
633+
634+ def test_evoloved_document_is_what_we_expect(self):
635+ DocumentEvolution.evolve_document(self.doc, one_step=True)
636+ fmt, evolved_doc = DocumentIO.load(
637+ resource_stream('linaro_dashboard_bundle',
638+ 'test_documents/evolution_1.5.json'),
639+ retain_order=False)
640+ self.assertEqual(self.doc, evolved_doc)
641+
642+
643+class DocumentEvolutionTests_1_5_to_1_6(TestCase):
644+
645+ def setUp(self):
646+ super(DocumentEvolutionTests_1_5_to_1_6, self).setUp()
647+ self.fmt, self.doc = DocumentIO.load(
648+ resource_stream('linaro_dashboard_bundle',
649+ 'test_documents/evolution_1.5.json'),
650+ retain_order=False)
651+
652+ def test_format_is_changed(self):
653+ self.assertEqual(self.doc["format"], "Dashboard Bundle Format 1.5")
654+ DocumentEvolution.evolve_document(self.doc, one_step=True)
655+ self.assertEqual(self.doc["format"], "Dashboard Bundle Format 1.6")
656+
657+ def test_evolved_document_is_latest_format(self):
658+ self.assertFalse(DocumentEvolution.is_latest(self.doc))
659+ DocumentEvolution.evolve_document(self.doc, one_step=True)
660 self.assertTrue(DocumentEvolution.is_latest(self.doc))
661
662 def test_evolved_document_is_valid(self):
663 DocumentEvolution.evolve_document(self.doc, one_step=True)
664 self.assertEqual(DocumentIO.check(self.doc),
665- "Dashboard Bundle Format 1.5")
666+ "Dashboard Bundle Format 1.6")
667
668 def test_evoloved_document_is_what_we_expect(self):
669 DocumentEvolution.evolve_document(self.doc, one_step=True)
670 fmt, evolved_doc = DocumentIO.load(
671 resource_stream('linaro_dashboard_bundle',
672- 'test_documents/evolution_1.5.json'),
673+ 'test_documents/evolution_1.6.json'),
674 retain_order=False)
675 self.assertEqual(self.doc, evolved_doc)

Subscribers

People subscribed via source and target branches