Merge lp:~milo/linaro-ci-dashboard/xml-to-dict into lp:linaro-ci-dashboard

Proposed by Milo Casagrande
Status: Superseded
Proposed branch: lp:~milo/linaro-ci-dashboard/xml-to-dict
Merge into: lp:linaro-ci-dashboard
Prerequisite: lp:~stevanr/linaro-ci-dashboard/build_results_xml
Diff against target: 810 lines (+669/-13) (has conflicts)
8 files modified
dashboard/frontend/android_textfield_loop/templates/android_textfield_loop_create.html (+0/-8)
dashboard/frontend/android_textfield_loop/tests/test_android_textfield_loop_model.py (+27/-2)
dashboard/frontend/models/loop_build.py (+13/-0)
dashboard/frontend/models/textfield_loop.py (+59/-3)
dashboard/frontend/tests/__init__.py (+3/-0)
dashboard/frontend/tests/test_xml_to_dict.py (+286/-0)
dashboard/lib/xml_fields.py (+51/-0)
dashboard/lib/xml_to_dict.py (+230/-0)
Text conflict in dashboard/frontend/android_textfield_loop/tests/test_android_textfield_loop_model.py
Text conflict in dashboard/frontend/models/textfield_loop.py
To merge this branch: bzr merge lp:~milo/linaro-ci-dashboard/xml-to-dict
Reviewer Review Type Date Requested Status
Stevan Radaković Pending
Данило Шеган Pending
Linaro Infrastructure Pending
Review via email: mp+122916@code.launchpad.net

Description of the change

Here there is all the logic for XML to dictionary and dictionary to XML conversions. Some tests and their values are provided.

To post a comment you must log in.
42. By Milo Casagrande

Removed print statements.

43. By Milo Casagrande

Fixed tests regressions, modified function.

44. By Milo Casagrande

Removed redundant errors div.

45. By Milo Casagrande

Merged from trunk.

46. By Milo Casagrande

Fixes for review.

47. By Milo Casagrande

Fixed assertion.

48. By Milo Casagrande

PEP8 fixes.

49. By Milo Casagrande

Added some tests.

50. By Milo Casagrande

Added note, fixed some tests.

51. By Milo Casagrande

Fixed reference.

52. By Milo Casagrande

Removed comment.

Unmerged revisions

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'dashboard/frontend/android_textfield_loop/templates/android_textfield_loop_create.html'
2--- dashboard/frontend/android_textfield_loop/templates/android_textfield_loop_create.html 2012-08-31 15:21:11 +0000
3+++ dashboard/frontend/android_textfield_loop/templates/android_textfield_loop_create.html 2012-09-06 08:33:21 +0000
4@@ -6,14 +6,6 @@
5 <form name="{{ form_name }}" action="{% url AndroidTextFieldLoopCreate %}" method="post">
6 {% csrf_token %}
7 {% endblock create_form %}
8-
9-{% if form.non_field_errors %}
10- <div class="form_error">
11- {% for err in form.non_field_errors %}
12- <div class="error_message">{{ err }}</div>
13- {% endfor %}
14- </div>
15-{% endif %}
16 {{ form.as_p }}
17 <div><input type="submit" value="Submit" /></div>
18 </form>
19
20=== modified file 'dashboard/frontend/android_textfield_loop/tests/test_android_textfield_loop_model.py'
21--- dashboard/frontend/android_textfield_loop/tests/test_android_textfield_loop_model.py 2012-09-05 13:59:57 +0000
22+++ dashboard/frontend/android_textfield_loop/tests/test_android_textfield_loop_model.py 2012-09-06 08:33:21 +0000
23@@ -24,9 +24,9 @@
24
25 A_NAME = 'a-build'
26 B_NAME = 'b-build'
27- VALID_VALUES = 'a=2\nb=3'
28+ VALID_VALUES = u'a=2\nb=3'
29 VALID_LINES = ['a=2', 'b=3']
30- NON_VALID_VALUES = 'a:2\nb=3'
31+ NON_VALID_VALUES = u'a:2\nb=3'
32 NON_VALID_LINES = ['a:2', 'b=3']
33 VALID_DICT = {'a': '2', 'b': '3'}
34
35@@ -52,6 +52,7 @@
36 def test_valid_values_wrong(self):
37 self.assertEqual((False, self.NON_VALID_LINES),
38 AndroidTextFieldLoop.valid_values(
39+<<<<<<< TREE
40 self.NON_VALID_VALUES))
41
42 def test_schedule_build(self):
43@@ -62,3 +63,27 @@
44 def test_schedule_build_invalid(self):
45 build = self.non_valid_android_loop.schedule_build()
46 self.assertEqual(build.result_xml, {})
47+=======
48+ self.NON_VALID_VALUES))
49+
50+ def test_schedule_build(self):
51+ build = self.android_loop.schedule_build()
52+ expected_out = '<?xml version="1.0" encoding="UTF-8"?>\n<!DOCTYPE ' \
53+ 'loop [\n <!ELEMENT loop (description?,fields)>\n' \
54+ ' <!ELEMENT description (#PCDATA)>\n ' \
55+ '<!ELEMENT fields (field+)>\n <!ELEMENT field ' \
56+ '(#PCDATA)>\n <!ATTLIST field name CDATA ' \
57+ '#REQUIRED>\n <!ATTLIST field type (text|int|bool) ' \
58+ '"text">\n]><loop><fields><field name="a">2</field>' \
59+ '<field name="b">3</field><field name="name">a-build' \
60+ '</field><field name="is_restricted">False</field>' \
61+ '<field name="is_official">False</field>' \
62+ '<field name="type">AndroidTextFieldLoop</field>' \
63+ '</fields></loop>'
64+ self.assertEqual(expected_out, build.result_xml)
65+ self.assertEqual(build.status, "success")
66+
67+ def test_schedule_build_invalid(self):
68+ build = self.non_valid_android_loop.schedule_build()
69+ self.assertEqual("", build.result_xml)
70+>>>>>>> MERGE-SOURCE
71
72=== modified file 'dashboard/frontend/models/loop_build.py'
73--- dashboard/frontend/models/loop_build.py 2012-09-05 13:23:05 +0000
74+++ dashboard/frontend/models/loop_build.py 2012-09-06 08:33:21 +0000
75@@ -18,6 +18,7 @@
76
77 from django.db import models
78 from frontend.models.loop import Loop
79+from dashboard.lib.xml_to_dict import XmlToDict
80
81
82 class LoopBuild(models.Model):
83@@ -47,3 +48,15 @@
84 else:
85 self.build_number = 1
86 super(LoopBuild, self).save(*args, **kwargs)
87+
88+ def get_build_result(self):
89+ """
90+ Returns a Python dictionary representation of the build XML stored.
91+
92+ :return A dictionary of the XML result.
93+ """
94+ dict_result = {}
95+ if self.result_xml:
96+ xml_to_dict = XmlToDict(self.result_xml)
97+ dict_result = xml_to_dict.tree_to_dict()
98+ return dict_result
99
100=== modified file 'dashboard/frontend/models/textfield_loop.py'
101--- dashboard/frontend/models/textfield_loop.py 2012-09-05 13:59:57 +0000
102+++ dashboard/frontend/models/textfield_loop.py 2012-09-06 08:33:21 +0000
103@@ -17,6 +17,7 @@
104
105 from django.db import models
106 from frontend.models.loop import Loop
107+from dashboard.lib.xml_to_dict import DictToXml
108
109
110 # Default delimiter to separate values from keys in the text field.
111@@ -40,6 +41,7 @@
112 values = models.TextField()
113
114 def schedule_build(self, parameters=None):
115+<<<<<<< TREE
116 from frontend.models.loop_build import LoopBuild
117 build = LoopBuild()
118 build.loop = self
119@@ -55,14 +57,40 @@
120 return build
121
122 def values_to_dict(self):
123+=======
124+ from frontend.models.loop_build import LoopBuild
125+ build = LoopBuild()
126+ build.loop = self
127+ build.duration = 0.00
128+
129+ try:
130+ build.result_xml = self.dict_to_xml()
131+ build.status = 'success'
132+ except:
133+ build.status = 'failure'
134+
135+ build.save()
136+ return build
137+
138+ def values_to_dict(self, valid=None, lines=None):
139+>>>>>>> MERGE-SOURCE
140 """
141 Returns a dictionary representation of the values inserted. The
142 key<>value pairs are split on DEFAULT_DELIMITER.
143
144- :return A dictionary of the values inserted.
145+ The default parameters are used for a second iteration of this function
146+ if we do not want to parse again the lines, but we only want to get the
147+ dictionary.
148+
149+ :param valid: If the lines are valid or not.
150+ :type valid bool
151+ :param lines: The list of lines in the text field.
152+ :type lines list
153+ :return A dictionary of the values inserted in the text field.
154 """
155 text_to_dict = {}
156- valid, lines = self.valid_values(self.values)
157+ if valid is None and lines is None:
158+ valid, lines = self.valid_values(self.values)
159
160 if valid:
161 for line in lines:
162@@ -71,6 +99,19 @@
163
164 return text_to_dict
165
166+ def _all_values_to_dict(self):
167+ """
168+ Returns a dict representation of the necessary fields for the loops.
169+
170+ :return A Python dictionary with the necessary fields.
171+ """
172+ # TODO need a better way to serialize the model
173+ all_values_dict = {'is_official': self.is_official,
174+ 'is_restricted': self.is_restricted,
175+ 'name': self.name,
176+ 'type': self.type}
177+ return all_values_dict
178+
179 @staticmethod
180 def valid_values(values):
181 """
182@@ -79,7 +120,7 @@
183
184 :param values: the string with all the key<>value pairs, separated with
185 a newline character.
186- :type str
187+ :type values unicode
188 :return a boolean for the validity, and the list of lines.
189 """
190 valid = True
191@@ -91,3 +132,18 @@
192 valid = False
193 break
194 return valid, lines
195+
196+ def dict_to_xml(self):
197+ """
198+ Converts the necessary values into an XML tree.
199+
200+ :return The XML tree as a string, or an empty string if the inserted
201+ values are not valid.
202+ """
203+ xml_string = ""
204+ valid, lines = self.valid_values(self.values)
205+ if valid:
206+ values = self.values_to_dict(valid, lines)
207+ values.update(self._all_values_to_dict())
208+ xml_string = DictToXml(values).dict_to_tree()
209+ return xml_string
210
211=== modified file 'dashboard/frontend/tests/__init__.py'
212--- dashboard/frontend/tests/__init__.py 2012-09-03 08:18:10 +0000
213+++ dashboard/frontend/tests/__init__.py 2012-09-06 08:33:21 +0000
214@@ -2,6 +2,7 @@
215 from dashboard.frontend.tests.test_models import *
216 from dashboard.frontend.tests.test_clientresponse import *
217 from dashboard.frontend.tests.test_custom_commands import *
218+from dashboard.frontend.tests.test_xml_to_dict import *
219
220
221 #starts the test suite
222@@ -14,4 +15,6 @@
223 'LoopTests': LoopTest,
224 'ClientResponseTests': ClientResponseTests,
225 'JenkinsCommandTest': JenkinsCommandTest,
226+ 'XmlToDictTest': XmlToDictTest,
227+ 'DictToXmlTest': DictToXmlTest,
228 }
229
230=== added file 'dashboard/frontend/tests/test_xml_to_dict.py'
231--- dashboard/frontend/tests/test_xml_to_dict.py 1970-01-01 00:00:00 +0000
232+++ dashboard/frontend/tests/test_xml_to_dict.py 2012-09-06 08:33:21 +0000
233@@ -0,0 +1,286 @@
234+# Copyright (C) 2012 Linaro
235+#
236+# This file is part of linaro-ci-dashboard.
237+#
238+# linaro-ci-dashboard is free software: you can redistribute it and/or modify
239+# it under the terms of the GNU Affero General Public License as published by
240+# the Free Software Foundation, either version 3 of the License, or
241+# (at your option) any later version.
242+#
243+# linaro-ci-dashboard is distributed in the hope that it will be useful,
244+# but WITHOUT ANY WARRANTY; without even the implied warranty of
245+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
246+# GNU Affero General Public License for more details.
247+#
248+# You should have received a copy of the GNU Affero General Public License
249+# along with linaro-ci-dashboard. If not, see <http://www.gnu.org/licenses/>.
250+
251+from django.test import TestCase
252+import os
253+from dashboard.lib.xml_to_dict import (
254+ XmlToDict,
255+ DictToXml
256+)
257+from dashboard.lib.xml_fields import (
258+ LOOP_ELEMENT,
259+ FIELDS_ELEMENT,
260+ TYPE_ATTR,
261+ NAME_ATTR,
262+ XML_START,
263+ INT_TYPE,
264+ BOOL_TYPE,
265+)
266+
267+
268+# XML tags
269+ROOT_TAG_OPEN = '<loop>'
270+ROOT_TAG_CLOSE = '</loop>'
271+FIELDS_TAG_OPEN = '<fields>'
272+FIELDS_TAG_CLOSE = '</fields>'
273+VALID_FIELD = '<field name="%(field_name)s">%(field_value)s</field>'
274+VALID_FIELD_WITH_TYPE = ('<field name="%(field_name)s" '
275+ 'type="%(field_type)s">%(field_value)s</field>')
276+
277+# Field names, types and values used as defaults for the tests.
278+LOOP_TYPE_VAL = 'text-build'
279+LOOP_NAME_VAL = 'build-test'
280+BUILD_TYPE = 'build_type'
281+BUILD_TYPE_VAL = 'build-android'
282+REPO_QUIET = 'repo_quiet'
283+REPO_QUIET_VAL = True
284+BUILD_FS_IMAGE = 'build_fs_image'
285+BUILD_FS_IMAGE_VAL = False
286+MAKE_JOBS = 'make_jobs'
287+MAKE_JOBS_VAL = 2
288+
289+NEWLINE = os.linesep
290+# Used to create the needed dictionary for fields creation.
291+# The dictionary has to be:
292+# {FIELD_NAME: value, FIELD_VALUE: value [, FIELD_TYPE: value]}
293+# FIELD_TYPE is optional.
294+FIELD_NAME = 'field_name'
295+FIELD_VALUE = 'field_value'
296+FIELD_TYPE = 'field_type'
297+
298+FIELDS = [
299+ {FIELD_NAME: BUILD_TYPE, FIELD_VALUE: BUILD_TYPE_VAL},
300+ {FIELD_NAME: REPO_QUIET, FIELD_TYPE: BOOL_TYPE,
301+ FIELD_VALUE: REPO_QUIET_VAL},
302+ {FIELD_NAME: BUILD_FS_IMAGE, FIELD_TYPE: BOOL_TYPE,
303+ FIELD_VALUE: BUILD_FS_IMAGE_VAL},
304+ {FIELD_NAME: MAKE_JOBS, FIELD_TYPE: INT_TYPE,
305+ FIELD_VALUE: MAKE_JOBS_VAL},
306+ {FIELD_NAME: NAME_ATTR, FIELD_VALUE: LOOP_NAME_VAL},
307+ {FIELD_NAME: TYPE_ATTR, FIELD_VALUE: LOOP_TYPE_VAL}
308+]
309+
310+VALID_START = XML_START + ROOT_TAG_OPEN
311+VALID_CLOSE = ROOT_TAG_CLOSE
312+
313+
314+def create_xml_fields(fields_list):
315+ """
316+ Support functions to create the <fields><field>... hierarchy.
317+
318+ :param fields_list: the list of fields to create. Each field has to be
319+ a dictionary with FIELD_NAME, FIELD_VALUE and optional FIELD_TYPE keys.
320+ :type fields_list list
321+ """
322+ assert isinstance(fields_list, list)
323+ fields_xml = FIELDS_TAG_OPEN
324+
325+ for field in fields_list:
326+ if field.get(FIELD_TYPE, None) is not None:
327+ fields_xml += VALID_FIELD_WITH_TYPE % field
328+ else:
329+ fields_xml += VALID_FIELD % field
330+ fields_xml += FIELDS_TAG_CLOSE
331+ return fields_xml
332+
333+
334+class XmlToDictTest(TestCase):
335+ """
336+ Test class for the conversion from XML to Python dictionary.
337+ The XML specification is taken from the HACKING file.
338+ """
339+
340+ def setUp(self):
341+ super(XmlToDictTest, self).setUp()
342+ self.xml_string = VALID_START
343+ self.xml_string += create_xml_fields(FIELDS)
344+ self.xml_string += VALID_CLOSE
345+ self.xml_to_dict = XmlToDict(self.xml_string)
346+
347+ def test_get_element_from_root(self):
348+ """
349+ Tests retrieval of the elements in the XML tree, but the root.
350+ """
351+ self.assertEqual(self.xml_to_dict.get_elements_from_root()[0].tag,
352+ FIELDS_ELEMENT)
353+
354+ def test_get_element_by_name_correct(self):
355+ """Tests retrieving of an element with a correct name."""
356+ self.assertEqual(FIELDS_ELEMENT,
357+ self.xml_to_dict.get_element_by_name(
358+ FIELDS_ELEMENT).tag)
359+
360+ def test_get_element_by_name_wrong(self):
361+ """ Tests retrieving of an element with a wrong name."""
362+ self.assertEqual(None,
363+ self.xml_to_dict.get_element_by_name('wrong'))
364+
365+ def test_elements_to_dict(self):
366+ """
367+ Tests creation of a dictionary with multiple elements.
368+ """
369+ expected_output = {MAKE_JOBS: MAKE_JOBS_VAL,
370+ BUILD_TYPE: BUILD_TYPE_VAL,
371+ BUILD_FS_IMAGE: BUILD_FS_IMAGE_VAL,
372+ REPO_QUIET: REPO_QUIET_VAL,
373+ NAME_ATTR: LOOP_NAME_VAL,
374+ TYPE_ATTR: LOOP_TYPE_VAL}
375+ element = self.xml_to_dict.get_element_by_name(FIELDS_ELEMENT)
376+ self.assertEqual(expected_output,
377+ self.xml_to_dict.element_to_dict(element))
378+
379+ def test_bool_true_conversion(self):
380+ """
381+ Tests the True bool conversion.
382+ """
383+ expected_out = {'a-field': True}
384+ for value in ['1', 1, 'y', 'Yes', True, 'True']:
385+ local_xml = VALID_START
386+ fields_list = [{FIELD_NAME: 'a-field',
387+ FIELD_TYPE: BOOL_TYPE,
388+ FIELD_VALUE: value}]
389+ local_xml += create_xml_fields(fields_list)
390+ local_xml += VALID_CLOSE
391+ local_xml_to_dict = XmlToDict(local_xml)
392+ element = local_xml_to_dict.get_element_by_name(FIELDS_ELEMENT)
393+ self.assertEqual(expected_out,
394+ local_xml_to_dict.element_to_dict(element))
395+
396+ def test_bool_false_conversion(self):
397+ """
398+ Tests the False bool conversion.
399+ """
400+ expected_out = {'a-field': False}
401+ for value in ['n', 0, 42, 'Yep', 'No', False, 'False', '?']:
402+ local_xml = VALID_START
403+ fields_list = [{FIELD_NAME: 'a-field',
404+ FIELD_TYPE: BOOL_TYPE,
405+ FIELD_VALUE: value}]
406+ local_xml += create_xml_fields(fields_list)
407+ local_xml += VALID_CLOSE
408+ local_xml_to_dict = XmlToDict(local_xml)
409+ element = local_xml_to_dict.get_element_by_name(FIELDS_ELEMENT)
410+ self.assertEqual(expected_out,
411+ local_xml_to_dict.element_to_dict(element))
412+
413+ def test_int_conversion(self):
414+ """
415+ Tests the int conversion.
416+ """
417+ expected_out = {'a-field': 42}
418+ for value in [42, '42', '42 ', ' 42']:
419+ local_xml = VALID_START
420+ fields_list = [{FIELD_NAME: 'a-field',
421+ FIELD_TYPE: INT_TYPE,
422+ FIELD_VALUE: value}]
423+ local_xml += create_xml_fields(fields_list)
424+ local_xml += VALID_CLOSE
425+ local_xml_to_dict = XmlToDict(local_xml)
426+ element = local_xml_to_dict.get_element_by_name(FIELDS_ELEMENT)
427+ self.assertEqual(expected_out,
428+ local_xml_to_dict.element_to_dict(element))
429+
430+ def test_str_conversion(self):
431+ """
432+ Tests the str conversion.
433+ """
434+ expected_out = {'a-field': '42'}
435+ for value in [42, '42', '42 ', ' 42']:
436+ local_xml = VALID_START
437+ fields_list = [{FIELD_NAME: 'a-field',
438+ FIELD_VALUE: value}]
439+ local_xml += create_xml_fields(fields_list)
440+ local_xml += VALID_CLOSE
441+ local_xml_to_dict = XmlToDict(local_xml)
442+ element = local_xml_to_dict.get_element_by_name(FIELDS_ELEMENT)
443+ self.assertEqual(expected_out,
444+ local_xml_to_dict.element_to_dict(element))
445+
446+ def test_tree_to_dict(self):
447+ """
448+ Tests the conversion of a dictionary into a full XML tree.
449+ """
450+ expected_output = {
451+ REPO_QUIET: REPO_QUIET_VAL,
452+ NAME_ATTR: LOOP_NAME_VAL,
453+ MAKE_JOBS: MAKE_JOBS_VAL,
454+ TYPE_ATTR: LOOP_TYPE_VAL,
455+ BUILD_TYPE: BUILD_TYPE_VAL,
456+ BUILD_FS_IMAGE: BUILD_FS_IMAGE_VAL
457+ }
458+ self.assertEqual(expected_output, self.xml_to_dict.tree_to_dict())
459+
460+ def test_get_root_name(self):
461+ """
462+ Tests retrieving of the root element.
463+ """
464+ self.assertEqual(self.xml_to_dict.get_root(), LOOP_ELEMENT)
465+
466+ def test_get_root_attributes(self):
467+ """
468+ Tests retrieving of the root attributes.
469+ """
470+ self.assertEqual({}, self.xml_to_dict.get_root_attributes())
471+
472+
473+class DictToXmlTest(TestCase):
474+ """
475+ Test class for the conversion from Python dictionary into XML.
476+ The XML specification is taken from the HACKING file.
477+ """
478+
479+ FIELDS_LIST = [
480+ {FIELD_NAME: MAKE_JOBS, FIELD_VALUE: MAKE_JOBS_VAL},
481+ {FIELD_NAME: TYPE_ATTR, FIELD_VALUE: LOOP_TYPE_VAL},
482+ {FIELD_NAME: NAME_ATTR, FIELD_VALUE: LOOP_NAME_VAL},
483+ ]
484+
485+ VALID_DICT = {
486+ NAME_ATTR: LOOP_NAME_VAL, TYPE_ATTR: LOOP_TYPE_VAL,
487+ MAKE_JOBS: str(MAKE_JOBS_VAL)
488+ }
489+
490+ def setUp(self):
491+ super(DictToXmlTest, self).setUp()
492+ self.valid_xml = VALID_START
493+ self.valid_xml += create_xml_fields(self.FIELDS_LIST)
494+ self.valid_xml += VALID_CLOSE
495+ self.dict_to_xml = DictToXml(self.VALID_DICT)
496+
497+ def test_dict_to_tree(self):
498+ """
499+ Tests dictionary to XML conversion.
500+ """
501+ self.assertEqual(self.valid_xml, self.dict_to_xml.dict_to_tree())
502+
503+ def test_dict_to_tree_with_bool(self):
504+ """
505+ Tests dictionary to XML conversion, with a bool inside, ElementTree
506+ does not cope very well with bool type.
507+ """
508+ fields = [
509+ {FIELD_NAME: REPO_QUIET, FIELD_VALUE: REPO_QUIET_VAL},
510+ {FIELD_NAME: TYPE_ATTR, FIELD_VALUE: LOOP_TYPE_VAL},
511+ {FIELD_NAME: NAME_ATTR, FIELD_VALUE: LOOP_NAME_VAL},
512+ ]
513+ dictionary = {TYPE_ATTR: LOOP_TYPE_VAL, NAME_ATTR: LOOP_NAME_VAL,
514+ REPO_QUIET: REPO_QUIET_VAL}
515+ dict_2_xml = DictToXml(dictionary)
516+ xml_string = VALID_START
517+ xml_string += create_xml_fields(fields)
518+ xml_string += VALID_CLOSE
519+ self.assertEqual(xml_string, dict_2_xml.dict_to_tree())
520
521=== added file 'dashboard/lib/xml_fields.py'
522--- dashboard/lib/xml_fields.py 1970-01-01 00:00:00 +0000
523+++ dashboard/lib/xml_fields.py 2012-09-06 08:33:21 +0000
524@@ -0,0 +1,51 @@
525+# Copyright (C) 2012 Linaro
526+#
527+# This file is part of linaro-ci-dashboard.
528+#
529+# linaro-ci-dashboard is free software: you can redistribute it and/or modify
530+# it under the terms of the GNU Affero General Public License as published by
531+# the Free Software Foundation, either version 3 of the License, or
532+# (at your option) any later version.
533+#
534+# linaro-ci-dashboard is distributed in the hope that it will be useful,
535+# but WITHOUT ANY WARRANTY; without even the implied warranty of
536+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
537+# GNU Affero General Public License for more details.
538+#
539+# You should have received a copy of the GNU Affero General Public License
540+# along with linaro-ci-dashboard. If not, see <http://www.gnu.org/licenses/>.
541+# Django settings for dashboard project.
542+
543+# The file contains all the necessary elements, tags and everything else
544+# related to the XML representation of a chain-build.
545+
546+# All the XML valid elements.
547+FIELD_ELEMENT = 'field'
548+FIELDS_ELEMENT = 'fields'
549+LOOP_ELEMENT = 'loop'
550+DESCRIPTION_ELEMENT = 'description'
551+
552+# All the XML attributes.
553+NAME_ATTR = 'name'
554+TYPE_ATTR = 'type'
555+
556+# The valid types for element values.
557+BOOL_TYPE = 'bool'
558+INT_TYPE = 'int'
559+STR_TYPE = 'text'
560+VALID_TYPES = [BOOL_TYPE, INT_TYPE, STR_TYPE]
561+
562+# List of valid values considered as boolean True.
563+VALID_TRUES = ['1', 'y', 'yes', 'Yes', 'True', 'true']
564+
565+# This is the start of the XML file. The DTD, if updated, needs to be updated
566+# also in the HACKING file. This one is taken from the HACKING file.
567+XML_START = '''<?xml version="1.0" encoding="UTF-8"?>
568+<!DOCTYPE loop [
569+ <!ELEMENT loop (description?,fields)>
570+ <!ELEMENT description (#PCDATA)>
571+ <!ELEMENT fields (field+)>
572+ <!ELEMENT field (#PCDATA)>
573+ <!ATTLIST field name CDATA #REQUIRED>
574+ <!ATTLIST field type (text|int|bool) "text">
575+]>'''
576
577=== added file 'dashboard/lib/xml_to_dict.py'
578--- dashboard/lib/xml_to_dict.py 1970-01-01 00:00:00 +0000
579+++ dashboard/lib/xml_to_dict.py 2012-09-06 08:33:21 +0000
580@@ -0,0 +1,230 @@
581+# Copyright (C) 2012 Linaro
582+#
583+# This file is part of linaro-ci-dashboard.
584+#
585+# linaro-ci-dashboard is free software: you can redistribute it and/or modify
586+# it under the terms of the GNU Affero General Public License as published by
587+# the Free Software Foundation, either version 3 of the License, or
588+# (at your option) any later version.
589+#
590+# linaro-ci-dashboard is distributed in the hope that it will be useful,
591+# but WITHOUT ANY WARRANTY; without even the implied warranty of
592+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
593+# GNU Affero General Public License for more details.
594+#
595+# You should have received a copy of the GNU Affero General Public License
596+# along with linaro-ci-dashboard. If not, see <http://www.gnu.org/licenses/>.
597+# Django settings for dashboard project.
598+
599+import xml.etree.ElementTree as ET
600+from dashboard.lib.xml_fields import *
601+
602+
603+class XmlToDict(object):
604+ """
605+ Class to convert an XML string into a Python dictionary.
606+ This applies to the XML as defined for the loop chaining.
607+
608+ Simple usage:
609+ xml_2_dict = XmlToDict(xml_string).tree_to_dict()
610+ """
611+
612+ def __init__(self, xml):
613+ """
614+ Initialize the XmlToDict class.
615+
616+ :param xml: The XML tree as a string.
617+ :type xml str
618+ """
619+ assert isinstance(xml, str)
620+
621+ super(XmlToDict, self).__init__()
622+ self.tree = ET.fromstring(xml)
623+
624+ def get_root(self):
625+ """
626+ Returns the name of the root of the XML tree.
627+ :return The name of the root element.
628+ """
629+ return self.tree.tag
630+
631+ def get_root_attributes(self):
632+ """
633+ Returns a dictionary containing the attributes of the root element.
634+
635+ :return A dictionary with the attributes of the root element.
636+ """
637+ root_attr = {}
638+ for item in self.tree.items():
639+ root_attr[item[0]] = item[1]
640+ return root_attr
641+
642+ def get_elements_from_root(self):
643+ """
644+ Returns the list of elements of the XML tree starting from the root.
645+ :return A list with Element instances.
646+ """
647+ return list(self.tree)
648+
649+ def get_element_by_name(self, name):
650+ """
651+ Returns an element from the XML tree by its name.
652+ :param name: the name of the element to get.
653+ :type name str
654+ :return An Element instance, None if not found.
655+ """
656+ return self.tree.find(name)
657+
658+ def element_to_dict(self, element):
659+ """
660+ Returns a dictionary representation of the provided XML element.
661+
662+ :param element: The Element to convert.
663+ :type element Element
664+ :return The dictionary representation of element.
665+ """
666+ assert isinstance(element, ET.Element)
667+
668+ element_dict = {}
669+ element_tag = element.tag
670+ element_text = element.text
671+
672+ if element_text:
673+ element_text = self._check_element_text(element_text)
674+
675+ attributes = element.items()
676+ children = list(element)
677+
678+ # Special case for XML 'field' tag.
679+ # 'field' tag must have a value and at least 'name' attribute set.
680+ if element_tag == FIELD_ELEMENT:
681+ if attributes:
682+ element_name = element.get(NAME_ATTR)
683+ element_type = element.get(TYPE_ATTR)
684+ # Convert the value into the specified type.
685+ value = self.convert_to_type(element_type, element.text)
686+ element_dict[element_name] = value
687+ else:
688+ if attributes:
689+ # Other case, we store all attributes as key<>value, plus also
690+ # the tag name and its value if OK.
691+ # There might be name collisions if XML is not well done.
692+ for attribute in attributes:
693+ element_dict[attribute[0]] = attribute[1]
694+ if element_text:
695+ element_dict[element_tag] = element_text
696+ elif element_text:
697+ element_dict[element_tag] = element_text
698+ if children:
699+ for child in children:
700+ element_dict.update(self.element_to_dict(child))
701+ return element_dict
702+
703+ def _check_element_text(self, text):
704+ """
705+ Check that the value contained in an XML element is valid, meaning it
706+ is not an empty space, a newline. In these cases, we return None since
707+ we do not want to have that element in the final dictionary.
708+
709+ :param text: The value contained between opening and closing XML tag.
710+ :type text str
711+ :return None if text is empty spaces or newlines, text otherwise.
712+ """
713+ # We do this since when we get nested elements, there might be 'valid'
714+ # characters, like newlines or empty spaces, between the opening and
715+ # closing tags, but we do not want to have such fields in the resulting
716+ # dictionary.
717+ import os
718+ text = str(text).strip()
719+ if text == os.linesep or len(text) == 0:
720+ text = None
721+ return text
722+
723+ def convert_to_type(self, type, value):
724+ """
725+ Converts a value of an XML element into the provided type. If type is
726+ None, tha value is converted into a string.
727+
728+ :param type: The type of the resulting value.
729+ :type type str
730+ :param value: The value associated with the XML element.
731+ :return The value converted into the specified type.
732+ """
733+ converted = str(value).strip()
734+ if type == BOOL_TYPE:
735+ if converted in VALID_TRUES:
736+ converted = True
737+ else:
738+ converted = False
739+ elif type == INT_TYPE:
740+ converted = int(converted)
741+
742+ return converted
743+
744+ def tree_to_dict(self):
745+ """
746+ Create a dictionary out of the XML tree.
747+
748+ :return A python dictionary of the XML tree.
749+ """
750+ tree_dict = {}
751+
752+ root_dict = self.get_root_attributes()
753+ elements_dict = {}
754+ for element in self.get_elements_from_root():
755+ elements_dict.update(self.element_to_dict(element))
756+
757+ if root_dict:
758+ tree_dict.update(root_dict)
759+
760+ if elements_dict:
761+ if tree_dict:
762+ tree_dict.update(elements_dict)
763+ else:
764+ tree_dict = elements_dict
765+
766+ return tree_dict
767+
768+
769+class DictToXml(object):
770+ """
771+ Class to convert a Python dictionary into an XML string .
772+
773+ Simple usage:
774+ dict_2_xml = DictToXml(dictionary).dict_to_tree()
775+ """
776+
777+ def __init__(self, dictionary):
778+ """
779+ Initialize the DictToXml class.
780+
781+ :param dictionary: The dictionary to convert.
782+ :type dictionary dict
783+ """
784+ assert isinstance(dictionary, dict)
785+ super(DictToXml, self).__init__()
786+ self.dictionary = dictionary
787+
788+ def dict_to_tree(self):
789+ """
790+ Create an XML tree out of the provided dictionary. The XML structure
791+ is defined in the HACKING file.
792+
793+ :return A string with the XML representation of the dictionary.
794+ """
795+ dict_to_xml = XML_START
796+
797+ loop = ET.Element(LOOP_ELEMENT)
798+ fields = ET.SubElement(loop, FIELDS_ELEMENT)
799+ for key, value in self.dictionary.iteritems():
800+ # TODO need to find a way to reflect the type of the data.
801+ attrib = {NAME_ATTR: key}
802+ field = ET.SubElement(fields, FIELD_ELEMENT, attrib)
803+ # ElementTree cannot serialize boolean types.
804+ field.text = str(value)
805+
806+ # Dump the XML and add it to the valid start with its DTD.
807+ xml_dump = ET.tostring(loop, encoding="utf-8")
808+ dict_to_xml += xml_dump
809+
810+ return dict_to_xml

Subscribers

People subscribed via source and target branches