Merge lp:~milo/linaro-ci-dashboard/xml-to-dict into lp:linaro-ci-dashboard
- xml-to-dict
- Merge into trunk
Status: | Merged |
---|---|
Approved by: | Stevan Radaković |
Approved revision: | 52 |
Merged at revision: | 41 |
Proposed branch: | lp:~milo/linaro-ci-dashboard/xml-to-dict |
Merge into: | lp:linaro-ci-dashboard |
Diff against target: |
960 lines (+722/-23) 13 files modified
HACKING (+1/-1) dashboard/frontend/android_textfield_loop/models/android_textfield_loop.py (+3/-0) dashboard/frontend/android_textfield_loop/templates/android_textfield_loop_create.html (+0/-8) dashboard/frontend/android_textfield_loop/tests/test_android_textfield_loop_model.py (+13/-4) dashboard/frontend/models/loop.py (+28/-4) dashboard/frontend/models/loop_build.py (+13/-0) dashboard/frontend/models/textfield_loop.py (+38/-5) dashboard/frontend/tests/__init__.py (+3/-0) dashboard/frontend/tests/test_models.py (+29/-1) dashboard/frontend/tests/test_xml_to_dict.py (+293/-0) dashboard/lib/template.py (+1/-0) dashboard/lib/xml_fields.py (+47/-0) dashboard/lib/xml_to_dict.py (+253/-0) |
To merge this branch: | bzr merge lp:~milo/linaro-ci-dashboard/xml-to-dict |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Stevan Radaković | Approve | ||
Linaro Infrastructure | Pending | ||
Данило Шеган | Pending | ||
Review via email:
|
Commit message
Description of the change
Here there is all the logic for XML to dictionary and dictionary to XML conversions. Some tests and their values are provided.

Milo Casagrande (milo) wrote : | # |
Hello Stevan,
thanks for the review!
On Thu, Sep 6, 2012 at 3:55 PM, Stevan Radaković
<email address hidden> wrote:
> Review: Needs Fixing
>
> Hey Milo great work !!1
> Before discussing the dict necessity with danilo, I'll do a review here and lets land this, with tendency to refactor things later.
>
>
>
> === modified file 'dashboard/
> --- dashboard/
> +++ dashboard/
> @@ -24,9 +24,9 @@
>
> A_NAME = 'a-build'
> B_NAME = 'b-build'
> - VALID_VALUES = 'a=2\nb=3'
> + VALID_VALUES = u'a=2\nb=3'
> VALID_LINES = ['a=2', 'b=3']
> - NON_VALID_VALUES = 'a:2\nb=3'
> + NON_VALID_VALUES = u'a:2\nb=3'
> NON_VALID_LINES = ['a:2', 'b=3']
> VALID_DICT = {'a': '2', 'b': '3'}
>
> @@ -52,6 +52,7 @@
> def test_valid_
> self.assertEqua
> AndroidTextFiel
> +<<<<<<< TREE
> self.NON_
>
> def test_schedule_
> @@ -62,3 +63,27 @@
> def test_schedule_
> build = self.non_
> self.assertEqua
> +=======
> + self.NON_
> +
> + def test_schedule_
> + build = self.android_
> + expected_out = '<?xml version="1.0" encoding=
> + 'loop [\n <!ELEMENT loop (description?
> + ' <!ELEMENT description (#PCDATA)>\n ' \
> + '<!ELEMENT fields (field+)>\n <!ELEMENT field ' \
> + '(#PCDATA)>\n <!ATTLIST field name CDATA ' \
> + '#REQUIRED>\n <!ATTLIST field type (text|int|bool) ' \
> + '"text"
> + '<field name="b"
> + '</field><field name="is_
> + '<field name="is_
> + '<field name="type"
> + '</fields></loop>'
> + self.assertEqua
> + self.assertEqua
> +
> + def test_schedule_
> + build = self.non_
> + self.assertEqua
> +>>>>>>> MERGE-SOURCE
>
>
> What's with all the white spaces in the expected out? Can't we do strip() somewhere in the code and output xml without new lines and white spaces?
The spaces are just in the DTD, that is a copy paste of the DTD as
defined in the HACKING file. That's the only part with spaces and
newlines. The XML output from the "parser" is just a one-...

Milo Casagrande (milo) wrote : | # |
On Fri, Sep 7, 2012 at 10:34 AM, Milo Casagrande
<email address hidden> wrote:
>
> The spaces are just in the DTD, that is a copy paste of the DTD as
> defined in the HACKING file. That's the only part with spaces and
> newlines. The XML output from the "parser" is just a one-liner.
Forgot to say that I will remove the spaces and newlines from the DTD
and make a one single line.
--
Milo Casagrande
Infrastructure Engineer
Linaro.org <www.linaro.org> │ Open source software for ARM SoCs

Stevan Radaković (stevanr) wrote : | # |
On 09/07/2012 10:43 AM, Milo Casagrande wrote:
> On Fri, Sep 7, 2012 at 10:34 AM, Milo Casagrande
> <email address hidden> wrote:
>> The spaces are just in the DTD, that is a copy paste of the DTD as
>> defined in the HACKING file. That's the only part with spaces and
>> newlines. The XML output from the "parser" is just a one-liner.
> Forgot to say that I will remove the spaces and newlines from the DTD
> and make a one single line.
>
hmm, you can use textwrap.

Stevan Radaković (stevanr) wrote : | # |
On 09/07/2012 10:37 AM, Milo Casagrande wrote:
> Hello Stevan,
>
> thanks for the review!
>
> On Thu, Sep 6, 2012 at 3:55 PM, Stevan Radaković
> <email address hidden> wrote:
>> Review: Needs Fixing
>>
>> Hey Milo great work !!1
>> Before discussing the dict necessity with danilo, I'll do a review here and lets land this, with tendency to refactor things later.
>>
>>
>>
>> === modified file 'dashboard/
>> --- dashboard/
>> +++ dashboard/
>> @@ -24,9 +24,9 @@
>>
>> A_NAME = 'a-build'
>> B_NAME = 'b-build'
>> - VALID_VALUES = 'a=2\nb=3'
>> + VALID_VALUES = u'a=2\nb=3'
>> VALID_LINES = ['a=2', 'b=3']
>> - NON_VALID_VALUES = 'a:2\nb=3'
>> + NON_VALID_VALUES = u'a:2\nb=3'
>> NON_VALID_LINES = ['a:2', 'b=3']
>> VALID_DICT = {'a': '2', 'b': '3'}
>>
>> @@ -52,6 +52,7 @@
>> def test_valid_
>> self.assertEqua
>> AndroidTextFiel
>> +<<<<<<< TREE
>> self.NON_
>>
>> def test_schedule_
>> @@ -62,3 +63,27 @@
>> def test_schedule_
>> build = self.non_
>> self.assertEqua
>> +=======
>> + self.NON_
>> +
>> + def test_schedule_
>> + build = self.android_
>> + expected_out = '<?xml version="1.0" encoding=
>> + 'loop [\n <!ELEMENT loop (description?
>> + ' <!ELEMENT description (#PCDATA)>\n ' \
>> + '<!ELEMENT fields (field+)>\n <!ELEMENT field ' \
>> + '(#PCDATA)>\n <!ATTLIST field name CDATA ' \
>> + '#REQUIRED>\n <!ATTLIST field type (text|int|bool) ' \
>> + '"text"
>> + '<field name="b"
>> + '</field><field name="is_
>> + '<field name="is_
>> + '<field name="type"
>> + '</fields></loop>'
>> + self.assertEqua
>> + self.assertEqua
>> +
>> + def test_schedule_
>> + build = self.non_
>> + self.assertEqua
>> +>>>>>>> MERGE-SOURCE
>>
>>
>> What's with all the white spaces in the expected out? Can't we do strip() somewhere in the code and output xml without new lines and white spaces?
> The spaces are just in the DTD, that is a copy paste of the DTD as
> de...
- 45. By Milo Casagrande
-
Merged from trunk.
- 46. By Milo Casagrande
-
Fixes for review.
- 47. By Milo Casagrande
-
Fixed assertion.
- 48. By Milo Casagrande
-
PEP8 fixes.
- 49. By Milo Casagrande
-
Added some tests.
- 50. By Milo Casagrande
-
Added note, fixed some tests.
- 51. By Milo Casagrande
-
Fixed reference.
- 52. By Milo Casagrande
-
Removed comment.

Milo Casagrande (milo) wrote : | # |
On Fri, Sep 7, 2012 at 12:28 PM, Stevan Radaković
<email address hidden> wrote:
>
> Good idea... We could also check if we can somehow make use of the
> 'exclude' option in the Meta subclass for filtering out the fields; or
> create our own option if it's not good match for us.
Unfortunately, we can't. The exclude option in the Meta class is only
available for the class that inhertis from Form, not for the models.
I pushed the last changes into the branch associated with this MP.
There is a "problem" also with using "model_to_dict": not all the
fields will be serialized to a dictionary. I wrote a note in the code.
The problem is with fields set as "editable=False". That parameter is
used with Form display, but since the "model_to_dict" is a function
taken from the "forms" part of Django, that parameter applies here as
well, so we do not get back everything, even if we include it in the
list of fields we want back.
I tried also with the serializers, but we would need to craft the
correct query, since the filter and exlude functions on serializers
need to have actual values to operate (we would end up doing
pk=self.pk, etc...).
The other options I see are or using getattr, or through
obj.__dict_
to pass a list of fields we want or we want to exclude.
Ciao.
--
Milo Casagrande
Infrastructure Engineer
Linaro.org <www.linaro.org> │ Open source software for ARM SoCs

Stevan Radaković (stevanr) wrote : | # |
Looks much better Milo.
Thanks for addressing all the comments and cleaning up things.
Approve +1.
Preview Diff
1 | === modified file 'HACKING' |
2 | --- HACKING 2012-09-05 16:50:12 +0000 |
3 | +++ HACKING 2012-09-07 12:52:19 +0000 |
4 | @@ -67,7 +67,7 @@ |
5 | <!ELEMENT fields (field+)> |
6 | <!ELEMENT field (#PCDATA)> |
7 | <!ATTLIST field name CDATA #REQUIRED> |
8 | - <!ATTLIST field type (text|int|bool) "text"> |
9 | + <!ATTLIST field type (str|int|bool) "str"> |
10 | ]> |
11 | <loop> |
12 | <fields> |
13 | |
14 | === modified file 'dashboard/frontend/android_textfield_loop/models/android_textfield_loop.py' |
15 | --- dashboard/frontend/android_textfield_loop/models/android_textfield_loop.py 2012-09-05 13:07:59 +0000 |
16 | +++ dashboard/frontend/android_textfield_loop/models/android_textfield_loop.py 2012-09-07 12:52:19 +0000 |
17 | @@ -23,6 +23,9 @@ |
18 | class Meta: |
19 | app_label = 'android_textfield_loop' |
20 | |
21 | + # Fields we want in the XML for chaining. |
22 | + XML_INCLUDE_LIST = ['name', 'is_official', 'is_restricted'] |
23 | + |
24 | def save(self, *args, **kwargs): |
25 | self.type = self.__class__.__name__ |
26 | super(self.__class__, self).save(*args, **kwargs) |
27 | |
28 | === modified file 'dashboard/frontend/android_textfield_loop/templates/android_textfield_loop_create.html' |
29 | --- dashboard/frontend/android_textfield_loop/templates/android_textfield_loop_create.html 2012-08-31 15:21:11 +0000 |
30 | +++ dashboard/frontend/android_textfield_loop/templates/android_textfield_loop_create.html 2012-09-07 12:52:19 +0000 |
31 | @@ -6,14 +6,6 @@ |
32 | <form name="{{ form_name }}" action="{% url AndroidTextFieldLoopCreate %}" method="post"> |
33 | {% csrf_token %} |
34 | {% endblock create_form %} |
35 | - |
36 | -{% if form.non_field_errors %} |
37 | - <div class="form_error"> |
38 | - {% for err in form.non_field_errors %} |
39 | - <div class="error_message">{{ err }}</div> |
40 | - {% endfor %} |
41 | - </div> |
42 | -{% endif %} |
43 | {{ form.as_p }} |
44 | <div><input type="submit" value="Submit" /></div> |
45 | </form> |
46 | |
47 | === modified file 'dashboard/frontend/android_textfield_loop/tests/test_android_textfield_loop_model.py' |
48 | --- dashboard/frontend/android_textfield_loop/tests/test_android_textfield_loop_model.py 2012-09-05 13:59:57 +0000 |
49 | +++ dashboard/frontend/android_textfield_loop/tests/test_android_textfield_loop_model.py 2012-09-07 12:52:19 +0000 |
50 | @@ -18,15 +18,16 @@ |
51 | from django.test import TestCase |
52 | from frontend.android_textfield_loop.models.android_textfield_loop \ |
53 | import AndroidTextFieldLoop |
54 | +from dashboard.lib.xml_fields import XML_START |
55 | |
56 | |
57 | class AndroidTextFieldLoopModelTest(TestCase): |
58 | |
59 | A_NAME = 'a-build' |
60 | B_NAME = 'b-build' |
61 | - VALID_VALUES = 'a=2\nb=3' |
62 | + VALID_VALUES = u'a=2\nb=3' |
63 | VALID_LINES = ['a=2', 'b=3'] |
64 | - NON_VALID_VALUES = 'a:2\nb=3' |
65 | + NON_VALID_VALUES = u'a:2\nb=3' |
66 | NON_VALID_LINES = ['a:2', 'b=3'] |
67 | VALID_DICT = {'a': '2', 'b': '3'} |
68 | |
69 | @@ -56,9 +57,17 @@ |
70 | |
71 | def test_schedule_build(self): |
72 | build = self.android_loop.schedule_build() |
73 | - self.assertEqual(build.result_xml, self.VALID_DICT) |
74 | + expected_out = (XML_START + |
75 | + '<loop><fields>' |
76 | + '<field name="a" type="str">2</field>' |
77 | + '<field name="is_restricted" type="bool">False</field>' |
78 | + '<field name="b" type="str">3</field>' |
79 | + '<field name="is_official" type="bool">False</field>' |
80 | + '<field name="name" type="str">a-build</field>' |
81 | + '</fields></loop>') |
82 | + self.assertEqual(expected_out, build.result_xml) |
83 | self.assertEqual(build.status, "success") |
84 | |
85 | def test_schedule_build_invalid(self): |
86 | build = self.non_valid_android_loop.schedule_build() |
87 | - self.assertEqual(build.result_xml, {}) |
88 | + self.assertEqual("", build.result_xml) |
89 | |
90 | === modified file 'dashboard/frontend/models/loop.py' |
91 | --- dashboard/frontend/models/loop.py 2012-09-04 14:51:52 +0000 |
92 | +++ dashboard/frontend/models/loop.py 2012-09-07 12:52:19 +0000 |
93 | @@ -44,7 +44,7 @@ |
94 | # List of fields to be excluded by default from the base64 encoded config. |
95 | # If a subclass needs different fields, it is necessary to override this |
96 | # variable. |
97 | - EXCLUDE_LIST = ['id', 'name', 'server', 'loop_ptr'] |
98 | + EXCLUDE_LIST = ['id', 'server', 'loop_ptr', 'next_loop', 'name', 'type'] |
99 | |
100 | def __init__(self, *args, **kwargs): |
101 | self.log = Logger.getClassLogger(self) |
102 | @@ -79,13 +79,37 @@ |
103 | pass |
104 | return build |
105 | |
106 | - def json(self): |
107 | + def json(self, include=None, exclude=None): |
108 | """Return Loop in a format suitable for serialization as JSON. The |
109 | - method is called without underscores to be callabel from Django |
110 | + method is called without underscores to be callable from Django |
111 | templates. |
112 | + |
113 | + :param include: the list of fields to be included |
114 | + :type include list |
115 | + :param exclude: the list of fields to be excluded |
116 | + :return A dictionary of the model fields |
117 | """ |
118 | from django.forms.models import model_to_dict |
119 | - return model_to_dict(self) |
120 | + # XXX |
121 | + # This method will not return all the fields, even if they are in the |
122 | + # include list. This is true for fields like 'type', that are set as |
123 | + # 'editable=False'. Since that is a parameter for the form, and we are |
124 | + # using a function from there, it applies here too. |
125 | + return model_to_dict(self, include, exclude) |
126 | + |
127 | + def dict_for_xml(self, include=None, exclude=None): |
128 | + """ |
129 | + Get a dictionary representation of this loop to be serialized into an |
130 | + XML form for chaining loop. |
131 | + |
132 | + :param include: the list of fields to be included |
133 | + :type include list |
134 | + :param exclude: the list of fields to be excluded |
135 | + :return A dictionary with the fields to be serialized. |
136 | + """ |
137 | + # Done in this way, calling self.json, since we have it, but if we |
138 | + # change the json representation, this needs to be changed as well. |
139 | + return self.json(include, exclude) |
140 | |
141 | @staticmethod |
142 | def can_chain_into(): |
143 | |
144 | === modified file 'dashboard/frontend/models/loop_build.py' |
145 | --- dashboard/frontend/models/loop_build.py 2012-09-05 13:23:05 +0000 |
146 | +++ dashboard/frontend/models/loop_build.py 2012-09-07 12:52:19 +0000 |
147 | @@ -18,6 +18,7 @@ |
148 | |
149 | from django.db import models |
150 | from frontend.models.loop import Loop |
151 | +from dashboard.lib.xml_to_dict import XmlToDict |
152 | |
153 | |
154 | class LoopBuild(models.Model): |
155 | @@ -47,3 +48,15 @@ |
156 | else: |
157 | self.build_number = 1 |
158 | super(LoopBuild, self).save(*args, **kwargs) |
159 | + |
160 | + def get_build_result(self): |
161 | + """ |
162 | + Returns a Python dictionary representation of the build XML stored. |
163 | + |
164 | + :return A dictionary of the XML result. |
165 | + """ |
166 | + dict_result = {} |
167 | + if self.result_xml: |
168 | + xml_to_dict = XmlToDict(self.result_xml) |
169 | + dict_result = xml_to_dict.tree_to_dict() |
170 | + return dict_result |
171 | |
172 | === modified file 'dashboard/frontend/models/textfield_loop.py' |
173 | --- dashboard/frontend/models/textfield_loop.py 2012-09-05 13:59:57 +0000 |
174 | +++ dashboard/frontend/models/textfield_loop.py 2012-09-07 12:52:19 +0000 |
175 | @@ -17,6 +17,7 @@ |
176 | |
177 | from django.db import models |
178 | from frontend.models.loop import Loop |
179 | +from dashboard.lib.xml_to_dict import DictToXml |
180 | |
181 | |
182 | # Default delimiter to separate values from keys in the text field. |
183 | @@ -39,6 +40,11 @@ |
184 | |
185 | values = models.TextField() |
186 | |
187 | + # List of fields to be included and excluded from serialization. |
188 | + # Each subclass should override these two variables. |
189 | + XML_INCLUDE_LIST = None |
190 | + XML_EXCLUDE_LIST = None |
191 | + |
192 | def schedule_build(self, parameters=None): |
193 | from frontend.models.loop_build import LoopBuild |
194 | build = LoopBuild() |
195 | @@ -46,7 +52,7 @@ |
196 | build.duration = 0.00 |
197 | |
198 | try: |
199 | - build.result_xml = self.values_to_dict() |
200 | + build.result_xml = self.values_to_xml() |
201 | build.status = 'success' |
202 | except: |
203 | build.status = 'failure' |
204 | @@ -54,15 +60,24 @@ |
205 | build.save() |
206 | return build |
207 | |
208 | - def values_to_dict(self): |
209 | + def values_to_dict(self, valid=None, lines=None): |
210 | """ |
211 | Returns a dictionary representation of the values inserted. The |
212 | key<>value pairs are split on DEFAULT_DELIMITER. |
213 | |
214 | - :return A dictionary of the values inserted. |
215 | + The default parameters are used for a second iteration of this function |
216 | + if we do not want to parse again the lines, but we only want to get the |
217 | + dictionary. |
218 | + |
219 | + :param valid: If the lines are valid or not. |
220 | + :type valid bool |
221 | + :param lines: The list of lines in the text field. |
222 | + :type lines list |
223 | + :return A dictionary of the values inserted in the text field. |
224 | """ |
225 | text_to_dict = {} |
226 | - valid, lines = self.valid_values(self.values) |
227 | + if valid is None and lines is None: |
228 | + valid, lines = self.valid_values(self.values) |
229 | |
230 | if valid: |
231 | for line in lines: |
232 | @@ -79,7 +94,7 @@ |
233 | |
234 | :param values: the string with all the key<>value pairs, separated with |
235 | a newline character. |
236 | - :type str |
237 | + :type values unicode |
238 | :return a boolean for the validity, and the list of lines. |
239 | """ |
240 | valid = True |
241 | @@ -91,3 +106,21 @@ |
242 | valid = False |
243 | break |
244 | return valid, lines |
245 | + |
246 | + def values_to_xml(self): |
247 | + """ |
248 | + Converts the necessary values into an XML tree. |
249 | + |
250 | + :return The XML tree as a string, or an empty string if the inserted |
251 | + values are not valid. |
252 | + """ |
253 | + xml_string = "" |
254 | + valid, lines = self.valid_values(self.values) |
255 | + if valid: |
256 | + values = self.values_to_dict(valid, lines) |
257 | + other_values = self.dict_for_xml(include=self.XML_INCLUDE_LIST, |
258 | + exclude=self.XML_EXCLUDE_LIST) |
259 | + if other_values: |
260 | + values.update(other_values) |
261 | + xml_string = DictToXml(values).dict_to_tree() |
262 | + return xml_string |
263 | |
264 | === modified file 'dashboard/frontend/tests/__init__.py' |
265 | --- dashboard/frontend/tests/__init__.py 2012-09-03 08:18:10 +0000 |
266 | +++ dashboard/frontend/tests/__init__.py 2012-09-07 12:52:19 +0000 |
267 | @@ -2,6 +2,7 @@ |
268 | from dashboard.frontend.tests.test_models import * |
269 | from dashboard.frontend.tests.test_clientresponse import * |
270 | from dashboard.frontend.tests.test_custom_commands import * |
271 | +from dashboard.frontend.tests.test_xml_to_dict import * |
272 | |
273 | |
274 | #starts the test suite |
275 | @@ -14,4 +15,6 @@ |
276 | 'LoopTests': LoopTest, |
277 | 'ClientResponseTests': ClientResponseTests, |
278 | 'JenkinsCommandTest': JenkinsCommandTest, |
279 | + 'XmlToDictTest': XmlToDictTest, |
280 | + 'DictToXmlTest': DictToXmlTest, |
281 | } |
282 | |
283 | === modified file 'dashboard/frontend/tests/test_models.py' |
284 | --- dashboard/frontend/tests/test_models.py 2012-09-04 11:56:22 +0000 |
285 | +++ dashboard/frontend/tests/test_models.py 2012-09-07 12:52:19 +0000 |
286 | @@ -26,6 +26,7 @@ |
287 | import AndroidTextFieldLoop |
288 | from lib.model_getter import ModelGetter |
289 | |
290 | + |
291 | class LoopTest(TestCase): |
292 | |
293 | def setUp(self): |
294 | @@ -42,6 +43,11 @@ |
295 | self.android_loop.build_type = "build-android" |
296 | self.android_loop.save() |
297 | |
298 | + self.loop = Loop() |
299 | + self.loop.name = 'a-loop' |
300 | + self.loop.type = Loop.__class__.__name__ |
301 | + self.loop.save() |
302 | + |
303 | def test_get_child_object(self): |
304 | loop = Loop.objects.get(id=1) |
305 | self.assertEqual("testjob_integration", loop.get_child_object().name) |
306 | @@ -49,7 +55,6 @@ |
307 | self.assertEqual("IntegrationLoop", |
308 | loop.get_child_object().__class__.__name__) |
309 | |
310 | - |
311 | def test_get_all_chainable_integration_empty(self): |
312 | chainable_loops = IntegrationLoop.get_all_chainable() |
313 | # Integration loop does not have any chainable loops |
314 | @@ -64,3 +69,26 @@ |
315 | def test_model_getter_unique_model(self): |
316 | all_models = [model.__name__ for model in get_models()] |
317 | self.assertTrue(len(all_models) == len(set(all_models))) |
318 | + |
319 | + def test_model_json(self): |
320 | + expected_out = {'next_loop': None, 'name': 'a-loop', 'server': None, |
321 | + 'is_restricted': False, 'is_official': False, 'id': 3} |
322 | + self.assertEqual(expected_out, self.loop.json()) |
323 | + |
324 | + def test_model_dict_for_xml(self): |
325 | + expected_out = {'next_loop': None, 'name': 'a-loop', 'server': None, |
326 | + 'is_restricted': False, 'is_official': False, 'id': 3} |
327 | + self.assertEqual(expected_out, self.loop.dict_for_xml()) |
328 | + |
329 | + def test_model_dict_for_xml_with_include(self): |
330 | + include_list = ['name', 'is_restricted'] |
331 | + expected_out = {'name': 'a-loop', 'is_restricted': False} |
332 | + self.assertEqual(expected_out, |
333 | + self.loop.dict_for_xml(include=include_list)) |
334 | + |
335 | + def test_model_dict_for_xml_with_exclude(self): |
336 | + exclude_list = ['is_restricted', 'name'] |
337 | + expected_out = {'next_loop': None, 'server': None, |
338 | + 'is_official': False, 'id': 3} |
339 | + self.assertEqual(expected_out, |
340 | + self.loop.dict_for_xml(exclude=exclude_list)) |
341 | |
342 | === added file 'dashboard/frontend/tests/test_xml_to_dict.py' |
343 | --- dashboard/frontend/tests/test_xml_to_dict.py 1970-01-01 00:00:00 +0000 |
344 | +++ dashboard/frontend/tests/test_xml_to_dict.py 2012-09-07 12:52:19 +0000 |
345 | @@ -0,0 +1,293 @@ |
346 | +# Copyright (C) 2012 Linaro |
347 | +# |
348 | +# This file is part of linaro-ci-dashboard. |
349 | +# |
350 | +# linaro-ci-dashboard is free software: you can redistribute it and/or modify |
351 | +# it under the terms of the GNU Affero General Public License as published by |
352 | +# the Free Software Foundation, either version 3 of the License, or |
353 | +# (at your option) any later version. |
354 | +# |
355 | +# linaro-ci-dashboard is distributed in the hope that it will be useful, |
356 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
357 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
358 | +# GNU Affero General Public License for more details. |
359 | +# |
360 | +# You should have received a copy of the GNU Affero General Public License |
361 | +# along with linaro-ci-dashboard. If not, see <http://www.gnu.org/licenses/>. |
362 | + |
363 | +from django.test import TestCase |
364 | +import os |
365 | +from dashboard.lib.xml_to_dict import ( |
366 | + XmlToDict, |
367 | + DictToXml |
368 | +) |
369 | +from dashboard.lib.xml_fields import ( |
370 | + LOOP_ELEMENT, |
371 | + FIELDS_ELEMENT, |
372 | + TYPE_ATTR, |
373 | + NAME_ATTR, |
374 | + XML_START, |
375 | + INT_TYPE, |
376 | + BOOL_TYPE, |
377 | + STR_TYPE, |
378 | +) |
379 | + |
380 | + |
381 | +# XML tags |
382 | +ROOT_TAG_OPEN = '<loop>' |
383 | +ROOT_TAG_CLOSE = '</loop>' |
384 | +FIELDS_TAG_OPEN = '<fields>' |
385 | +FIELDS_TAG_CLOSE = '</fields>' |
386 | +VALID_FIELD = '<field name="%(field_name)s">%(field_value)s</field>' |
387 | +VALID_FIELD_WITH_TYPE = ('<field name="%(field_name)s" ' |
388 | + 'type="%(field_type)s">%(field_value)s</field>') |
389 | + |
390 | +# Field names, types and values used as defaults for the tests. |
391 | +LOOP_TYPE_VAL = 'text-build' |
392 | +LOOP_NAME_VAL = 'build-test' |
393 | +BUILD_TYPE = 'build_type' |
394 | +BUILD_TYPE_VAL = 'build-android' |
395 | +REPO_QUIET = 'repo_quiet' |
396 | +REPO_QUIET_VAL = True |
397 | +BUILD_FS_IMAGE = 'build_fs_image' |
398 | +BUILD_FS_IMAGE_VAL = False |
399 | +MAKE_JOBS = 'make_jobs' |
400 | +MAKE_JOBS_VAL = 2 |
401 | + |
402 | +NEWLINE = os.linesep |
403 | +# Used to create the needed dictionary for fields creation. |
404 | +# The dictionary has to be: |
405 | +# {FIELD_NAME: value, FIELD_VALUE: value [, FIELD_TYPE: value]} |
406 | +# FIELD_TYPE is optional. |
407 | +FIELD_NAME = 'field_name' |
408 | +FIELD_VALUE = 'field_value' |
409 | +FIELD_TYPE = 'field_type' |
410 | + |
411 | +FIELDS = [ |
412 | + {FIELD_NAME: BUILD_TYPE, FIELD_VALUE: BUILD_TYPE_VAL}, |
413 | + {FIELD_NAME: REPO_QUIET, FIELD_TYPE: BOOL_TYPE, |
414 | + FIELD_VALUE: REPO_QUIET_VAL}, |
415 | + {FIELD_NAME: BUILD_FS_IMAGE, FIELD_TYPE: BOOL_TYPE, |
416 | + FIELD_VALUE: BUILD_FS_IMAGE_VAL}, |
417 | + {FIELD_NAME: MAKE_JOBS, FIELD_TYPE: INT_TYPE, |
418 | + FIELD_VALUE: MAKE_JOBS_VAL}, |
419 | + {FIELD_NAME: NAME_ATTR, FIELD_VALUE: LOOP_NAME_VAL}, |
420 | + {FIELD_NAME: TYPE_ATTR, FIELD_VALUE: LOOP_TYPE_VAL} |
421 | +] |
422 | + |
423 | +VALID_START = XML_START + ROOT_TAG_OPEN |
424 | +VALID_CLOSE = ROOT_TAG_CLOSE |
425 | + |
426 | + |
427 | +def create_xml_fields(fields_list): |
428 | + """ |
429 | + Support functions to create the <fields><field>... hierarchy. |
430 | + |
431 | + :param fields_list: the list of fields to create. Each field has to be |
432 | + a dictionary with FIELD_NAME, FIELD_VALUE and optional FIELD_TYPE keys. |
433 | + :type fields_list list |
434 | + """ |
435 | + assert isinstance(fields_list, list) |
436 | + fields_xml = FIELDS_TAG_OPEN |
437 | + |
438 | + for field in fields_list: |
439 | + if field.get(FIELD_TYPE, None) is not None: |
440 | + fields_xml += VALID_FIELD_WITH_TYPE % field |
441 | + else: |
442 | + fields_xml += VALID_FIELD % field |
443 | + fields_xml += FIELDS_TAG_CLOSE |
444 | + return fields_xml |
445 | + |
446 | + |
447 | +class XmlToDictTest(TestCase): |
448 | + """ |
449 | + Test class for the conversion from XML to Python dictionary. |
450 | + The XML specification is taken from the HACKING file. |
451 | + """ |
452 | + |
453 | + def setUp(self): |
454 | + super(XmlToDictTest, self).setUp() |
455 | + self.xml_string = VALID_START |
456 | + self.xml_string += create_xml_fields(FIELDS) |
457 | + self.xml_string += VALID_CLOSE |
458 | + self.xml_to_dict = XmlToDict(self.xml_string) |
459 | + |
460 | + def test_get_element_from_root(self): |
461 | + """ |
462 | + Tests retrieval of the elements in the XML tree, but the root. |
463 | + """ |
464 | + self.assertEqual(self.xml_to_dict.get_elements_from_root()[0].tag, |
465 | + FIELDS_ELEMENT) |
466 | + |
467 | + def test_get_element_by_name_correct(self): |
468 | + """Tests retrieving of an element with a correct name.""" |
469 | + self.assertEqual(FIELDS_ELEMENT, |
470 | + self.xml_to_dict.get_element_by_name( |
471 | + FIELDS_ELEMENT).tag) |
472 | + |
473 | + def test_get_element_by_name_wrong(self): |
474 | + """ Tests retrieving of an element with a wrong name.""" |
475 | + self.assertEqual(None, |
476 | + self.xml_to_dict.get_element_by_name('wrong')) |
477 | + |
478 | + def test_elements_to_dict(self): |
479 | + """ |
480 | + Tests creation of a dictionary with multiple elements. |
481 | + """ |
482 | + expected_output = {MAKE_JOBS: MAKE_JOBS_VAL, |
483 | + BUILD_TYPE: BUILD_TYPE_VAL, |
484 | + BUILD_FS_IMAGE: BUILD_FS_IMAGE_VAL, |
485 | + REPO_QUIET: REPO_QUIET_VAL, |
486 | + NAME_ATTR: LOOP_NAME_VAL, |
487 | + TYPE_ATTR: LOOP_TYPE_VAL} |
488 | + element = self.xml_to_dict.get_element_by_name(FIELDS_ELEMENT) |
489 | + self.assertEqual(expected_output, |
490 | + self.xml_to_dict.element_to_dict(element)) |
491 | + |
492 | + def test_bool_true_conversion(self): |
493 | + """ |
494 | + Tests the True bool conversion. |
495 | + """ |
496 | + expected_out = {'a-field': True} |
497 | + for value in ['1', 1, 'y', 'Yes', True, 'True']: |
498 | + local_xml = VALID_START |
499 | + fields_list = [{FIELD_NAME: 'a-field', |
500 | + FIELD_TYPE: BOOL_TYPE, |
501 | + FIELD_VALUE: value}] |
502 | + local_xml += create_xml_fields(fields_list) |
503 | + local_xml += VALID_CLOSE |
504 | + local_xml_to_dict = XmlToDict(local_xml) |
505 | + element = local_xml_to_dict.get_element_by_name(FIELDS_ELEMENT) |
506 | + self.assertEqual(expected_out, |
507 | + local_xml_to_dict.element_to_dict(element)) |
508 | + |
509 | + def test_bool_false_conversion(self): |
510 | + """ |
511 | + Tests the False bool conversion. |
512 | + """ |
513 | + expected_out = {'a-field': False} |
514 | + for value in ['n', 0, 42, 'Yep', 'No', False, 'False', '?']: |
515 | + local_xml = VALID_START |
516 | + fields_list = [{FIELD_NAME: 'a-field', |
517 | + FIELD_TYPE: BOOL_TYPE, |
518 | + FIELD_VALUE: value}] |
519 | + local_xml += create_xml_fields(fields_list) |
520 | + local_xml += VALID_CLOSE |
521 | + local_xml_to_dict = XmlToDict(local_xml) |
522 | + element = local_xml_to_dict.get_element_by_name(FIELDS_ELEMENT) |
523 | + self.assertEqual(expected_out, |
524 | + local_xml_to_dict.element_to_dict(element)) |
525 | + |
526 | + def test_int_conversion(self): |
527 | + """ |
528 | + Tests the int conversion. |
529 | + """ |
530 | + expected_out = {'a-field': 42} |
531 | + for value in [42, '42', '42 ', ' 42']: |
532 | + local_xml = VALID_START |
533 | + fields_list = [{FIELD_NAME: 'a-field', |
534 | + FIELD_TYPE: INT_TYPE, |
535 | + FIELD_VALUE: value}] |
536 | + local_xml += create_xml_fields(fields_list) |
537 | + local_xml += VALID_CLOSE |
538 | + local_xml_to_dict = XmlToDict(local_xml) |
539 | + element = local_xml_to_dict.get_element_by_name(FIELDS_ELEMENT) |
540 | + self.assertEqual(expected_out, |
541 | + local_xml_to_dict.element_to_dict(element)) |
542 | + |
543 | + def test_str_conversion(self): |
544 | + """ |
545 | + Tests the str conversion. |
546 | + """ |
547 | + expected_out = {'a-field': '42'} |
548 | + for value in [42, '42', '42 ', ' 42']: |
549 | + local_xml = VALID_START |
550 | + fields_list = [{FIELD_NAME: 'a-field', |
551 | + FIELD_VALUE: value}] |
552 | + local_xml += create_xml_fields(fields_list) |
553 | + local_xml += VALID_CLOSE |
554 | + local_xml_to_dict = XmlToDict(local_xml) |
555 | + element = local_xml_to_dict.get_element_by_name(FIELDS_ELEMENT) |
556 | + self.assertEqual(expected_out, |
557 | + local_xml_to_dict.element_to_dict(element)) |
558 | + |
559 | + def test_tree_to_dict(self): |
560 | + """ |
561 | + Tests the conversion of a dictionary into a full XML tree. |
562 | + """ |
563 | + expected_output = { |
564 | + REPO_QUIET: REPO_QUIET_VAL, |
565 | + NAME_ATTR: LOOP_NAME_VAL, |
566 | + MAKE_JOBS: MAKE_JOBS_VAL, |
567 | + TYPE_ATTR: LOOP_TYPE_VAL, |
568 | + BUILD_TYPE: BUILD_TYPE_VAL, |
569 | + BUILD_FS_IMAGE: BUILD_FS_IMAGE_VAL |
570 | + } |
571 | + self.assertEqual(expected_output, self.xml_to_dict.tree_to_dict()) |
572 | + |
573 | + def test_get_root_name(self): |
574 | + """ |
575 | + Tests retrieving of the root element. |
576 | + """ |
577 | + self.assertEqual(self.xml_to_dict.get_root(), LOOP_ELEMENT) |
578 | + |
579 | + def test_get_root_attributes(self): |
580 | + """ |
581 | + Tests retrieving of the root attributes. |
582 | + """ |
583 | + self.assertEqual({}, self.xml_to_dict.get_root_attributes()) |
584 | + |
585 | + |
586 | +class DictToXmlTest(TestCase): |
587 | + """ |
588 | + Test class for the conversion from Python dictionary into XML. |
589 | + The XML specification is taken from the HACKING file. |
590 | + """ |
591 | + |
592 | + FIELDS_LIST = [ |
593 | + {FIELD_NAME: MAKE_JOBS, FIELD_VALUE: MAKE_JOBS_VAL, |
594 | + FIELD_TYPE: INT_TYPE}, |
595 | + {FIELD_NAME: TYPE_ATTR, FIELD_VALUE: LOOP_TYPE_VAL, |
596 | + FIELD_TYPE: STR_TYPE}, |
597 | + {FIELD_NAME: NAME_ATTR, FIELD_VALUE: LOOP_NAME_VAL, |
598 | + FIELD_TYPE: STR_TYPE}, |
599 | + ] |
600 | + |
601 | + VALID_DICT = { |
602 | + NAME_ATTR: LOOP_NAME_VAL, TYPE_ATTR: LOOP_TYPE_VAL, |
603 | + MAKE_JOBS: MAKE_JOBS_VAL |
604 | + } |
605 | + |
606 | + def setUp(self): |
607 | + super(DictToXmlTest, self).setUp() |
608 | + self.valid_xml = VALID_START |
609 | + self.valid_xml += create_xml_fields(self.FIELDS_LIST) |
610 | + self.valid_xml += VALID_CLOSE |
611 | + self.dict_to_xml = DictToXml(self.VALID_DICT) |
612 | + |
613 | + def test_dict_to_tree(self): |
614 | + """ |
615 | + Tests dictionary to XML conversion. |
616 | + """ |
617 | + self.assertEqual(self.valid_xml, self.dict_to_xml.dict_to_tree()) |
618 | + |
619 | + def test_dict_to_tree_with_bool(self): |
620 | + """ |
621 | + Tests dictionary to XML conversion, with a bool inside, ElementTree |
622 | + does not cope very well with bool type. |
623 | + """ |
624 | + fields = [ |
625 | + {FIELD_NAME: REPO_QUIET, FIELD_VALUE: REPO_QUIET_VAL, |
626 | + FIELD_TYPE: BOOL_TYPE}, |
627 | + {FIELD_NAME: TYPE_ATTR, FIELD_VALUE: LOOP_TYPE_VAL, |
628 | + FIELD_TYPE: STR_TYPE}, |
629 | + {FIELD_NAME: NAME_ATTR, FIELD_VALUE: LOOP_NAME_VAL, |
630 | + FIELD_TYPE: STR_TYPE}, |
631 | + ] |
632 | + dictionary = {TYPE_ATTR: LOOP_TYPE_VAL, NAME_ATTR: LOOP_NAME_VAL, |
633 | + REPO_QUIET: REPO_QUIET_VAL} |
634 | + dict_2_xml = DictToXml(dictionary) |
635 | + xml_string = VALID_START |
636 | + xml_string += create_xml_fields(fields) |
637 | + xml_string += VALID_CLOSE |
638 | + self.assertEqual(xml_string, dict_2_xml.dict_to_tree()) |
639 | |
640 | === modified file 'dashboard/lib/template.py' |
641 | --- dashboard/lib/template.py 2012-08-22 14:44:49 +0000 |
642 | +++ dashboard/lib/template.py 2012-09-07 12:52:19 +0000 |
643 | @@ -16,6 +16,7 @@ |
644 | # along with linaro-ci-dashboard. If not, see <http://www.gnu.org/licenses/>. |
645 | # Django settings for dashboard project. |
646 | |
647 | + |
648 | class AbstractTemplate(object): |
649 | "Template class interface." |
650 | |
651 | |
652 | === added file 'dashboard/lib/xml_fields.py' |
653 | --- dashboard/lib/xml_fields.py 1970-01-01 00:00:00 +0000 |
654 | +++ dashboard/lib/xml_fields.py 2012-09-07 12:52:19 +0000 |
655 | @@ -0,0 +1,47 @@ |
656 | +# Copyright (C) 2012 Linaro |
657 | +# |
658 | +# This file is part of linaro-ci-dashboard. |
659 | +# |
660 | +# linaro-ci-dashboard is free software: you can redistribute it and/or modify |
661 | +# it under the terms of the GNU Affero General Public License as published by |
662 | +# the Free Software Foundation, either version 3 of the License, or |
663 | +# (at your option) any later version. |
664 | +# |
665 | +# linaro-ci-dashboard is distributed in the hope that it will be useful, |
666 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
667 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
668 | +# GNU Affero General Public License for more details. |
669 | +# |
670 | +# You should have received a copy of the GNU Affero General Public License |
671 | +# along with linaro-ci-dashboard. If not, see <http://www.gnu.org/licenses/>. |
672 | +# Django settings for dashboard project. |
673 | + |
674 | +# The file contains all the necessary elements, tags and everything else |
675 | +# related to the XML representation of a chain-build. |
676 | + |
677 | +# All the XML valid elements. |
678 | +FIELD_ELEMENT = 'field' |
679 | +FIELDS_ELEMENT = 'fields' |
680 | +LOOP_ELEMENT = 'loop' |
681 | +DESCRIPTION_ELEMENT = 'description' |
682 | + |
683 | +# All the XML attributes. |
684 | +NAME_ATTR = 'name' |
685 | +TYPE_ATTR = 'type' |
686 | + |
687 | +# The valid types for element values. |
688 | +BOOL_TYPE = 'bool' |
689 | +INT_TYPE = 'int' |
690 | +STR_TYPE = 'str' |
691 | +VALID_TYPES = [BOOL_TYPE, INT_TYPE, STR_TYPE] |
692 | + |
693 | +# List of valid values considered as boolean True. |
694 | +VALID_TRUES = ['1', 'y', 'yes', 'Yes', 'True', 'true'] |
695 | + |
696 | +# This is the start of the XML file. The DTD, if updated, needs to be updated |
697 | +# also in the HACKING file. This one is taken from the HACKING file. |
698 | +XML_START = ('<?xml version="1.0" encoding="UTF-8"?><!DOCTYPE loop ' |
699 | + '[<!ELEMENT loop (description?,fields)><!ELEMENT description ' |
700 | + '(#PCDATA)><!ELEMENT fields (field+)><!ELEMENT field ' |
701 | + '(#PCDATA)><!ATTLIST field name CDATA #REQUIRED>' |
702 | + '<!ATTLIST field type (str|int|bool) "str">]>') |
703 | |
704 | === added file 'dashboard/lib/xml_to_dict.py' |
705 | --- dashboard/lib/xml_to_dict.py 1970-01-01 00:00:00 +0000 |
706 | +++ dashboard/lib/xml_to_dict.py 2012-09-07 12:52:19 +0000 |
707 | @@ -0,0 +1,253 @@ |
708 | +# Copyright (C) 2012 Linaro |
709 | +# |
710 | +# This file is part of linaro-ci-dashboard. |
711 | +# |
712 | +# linaro-ci-dashboard is free software: you can redistribute it and/or modify |
713 | +# it under the terms of the GNU Affero General Public License as published by |
714 | +# the Free Software Foundation, either version 3 of the License, or |
715 | +# (at your option) any later version. |
716 | +# |
717 | +# linaro-ci-dashboard is distributed in the hope that it will be useful, |
718 | +# but WITHOUT ANY WARRANTY; without even the implied warranty of |
719 | +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the |
720 | +# GNU Affero General Public License for more details. |
721 | +# |
722 | +# You should have received a copy of the GNU Affero General Public License |
723 | +# along with linaro-ci-dashboard. If not, see <http://www.gnu.org/licenses/>. |
724 | +# Django settings for dashboard project. |
725 | + |
726 | +import xml.etree.ElementTree as ET |
727 | +from dashboard.lib.xml_fields import * |
728 | + |
729 | + |
730 | +class XmlToDictException(Exception): |
731 | + """ |
732 | + Exception for the XmlToDict class. |
733 | + """ |
734 | + |
735 | + |
736 | +class DictToXmlException(Exception): |
737 | + """ |
738 | + Exception fro the DictToXml class. |
739 | + """ |
740 | + |
741 | + |
742 | +class XmlToDict(object): |
743 | + """ |
744 | + Class to convert an XML string into a Python dictionary. |
745 | + This applies to the XML as defined for the loop chaining. |
746 | + |
747 | + Simple usage: |
748 | + xml_2_dict = XmlToDict(xml_string).tree_to_dict() |
749 | + """ |
750 | + |
751 | + def __init__(self, xml): |
752 | + """ |
753 | + Initialize the XmlToDict class. |
754 | + |
755 | + :param xml: The XML tree as a string. |
756 | + :type xml str |
757 | + """ |
758 | + if not isinstance(xml, str): |
759 | + raise XmlToDictException("Type of parameter is not 'str', " |
760 | + "got '%s' instead." % type(xml).__name__) |
761 | + super(XmlToDict, self).__init__() |
762 | + self.tree = ET.fromstring(xml) |
763 | + |
764 | + def get_root(self): |
765 | + """ |
766 | + Returns the name of the root of the XML tree. |
767 | + :return The name of the root element. |
768 | + """ |
769 | + return self.tree.tag |
770 | + |
771 | + def get_root_attributes(self): |
772 | + """ |
773 | + Returns a dictionary containing the attributes of the root element. |
774 | + |
775 | + :return A dictionary with the attributes of the root element. |
776 | + """ |
777 | + root_attr = {} |
778 | + for item in self.tree.items(): |
779 | + root_attr[item[0]] = item[1] |
780 | + return root_attr |
781 | + |
782 | + def get_elements_from_root(self): |
783 | + """ |
784 | + Returns the list of elements of the XML tree starting from the root. |
785 | + :return A list with Element instances. |
786 | + """ |
787 | + return list(self.tree) |
788 | + |
789 | + def get_element_by_name(self, name): |
790 | + """ |
791 | + Returns an element from the XML tree by its name. |
792 | + :param name: the name of the element to get. |
793 | + :type name str |
794 | + :return An Element instance, None if not found. |
795 | + """ |
796 | + return self.tree.find(name) |
797 | + |
798 | + def element_to_dict(self, element): |
799 | + """ |
800 | + Returns a dictionary representation of the provided XML element. |
801 | + |
802 | + :param element: The Element to convert. |
803 | + :type element Element |
804 | + :return The dictionary representation of element. |
805 | + """ |
806 | + if not isinstance(element, ET.Element): |
807 | + raise XmlToDictException("Type of parameter is not 'Element', " |
808 | + "got '%s' instead" % |
809 | + type(element).__name__) |
810 | + element_dict = {} |
811 | + element_tag = element.tag |
812 | + element_text = element.text |
813 | + |
814 | + if element_text: |
815 | + element_text = self._check_element_text(element_text) |
816 | + |
817 | + attributes = element.items() |
818 | + children = list(element) |
819 | + |
820 | + # Special case for XML 'field' tag. |
821 | + # 'field' tag must have a value and at least 'name' attribute set. |
822 | + if element_tag == FIELD_ELEMENT: |
823 | + if attributes: |
824 | + element_name = element.get(NAME_ATTR) |
825 | + element_type = element.get(TYPE_ATTR) |
826 | + # Convert the value into the specified type. |
827 | + value = self.convert_to_type(element_type, element.text) |
828 | + element_dict[element_name] = value |
829 | + else: |
830 | + if attributes: |
831 | + # Other case, we store all attributes as key<>value, plus also |
832 | + # the tag name and its value if OK. |
833 | + # There might be name collisions if XML is not well done. |
834 | + for attribute in attributes: |
835 | + element_dict[attribute[0]] = attribute[1] |
836 | + if element_text: |
837 | + element_dict[element_tag] = element_text |
838 | + elif element_text: |
839 | + element_dict[element_tag] = element_text |
840 | + if children: |
841 | + for child in children: |
842 | + element_dict.update(self.element_to_dict(child)) |
843 | + return element_dict |
844 | + |
845 | + def _check_element_text(self, text): |
846 | + """ |
847 | + Check that the value contained in an XML element is valid, meaning it |
848 | + is not an empty space, a newline. In these cases, we return None since |
849 | + we do not want to have that element in the final dictionary. |
850 | + |
851 | + :param text: The value contained between opening and closing XML tag. |
852 | + :type text str |
853 | + :return None if text is empty spaces or newlines, text otherwise. |
854 | + """ |
855 | + # We do this since when we get nested elements, there might be 'valid' |
856 | + # characters, like newlines or empty spaces, between the opening and |
857 | + # closing tags, but we do not want to have such fields in the resulting |
858 | + # dictionary. |
859 | + import os |
860 | + text = str(text).strip() |
861 | + if text == os.linesep or len(text) == 0: |
862 | + text = None |
863 | + return text |
864 | + |
865 | + def convert_to_type(self, value_type, value): |
866 | + """ |
867 | + Converts a value of an XML element into the provided type. If type is |
868 | + None, tha value is converted into a string. |
869 | + |
870 | + :param value_type: The type of the resulting value. |
871 | + :type value_type str |
872 | + :param value: The value associated with the XML element. |
873 | + :return The value converted into the specified type. |
874 | + """ |
875 | + converted = str(value).strip() |
876 | + if value_type == BOOL_TYPE: |
877 | + if converted in VALID_TRUES: |
878 | + converted = True |
879 | + else: |
880 | + converted = False |
881 | + elif value_type == INT_TYPE: |
882 | + converted = int(converted) |
883 | + |
884 | + return converted |
885 | + |
886 | + def tree_to_dict(self): |
887 | + """ |
888 | + Create a dictionary out of the XML tree. |
889 | + |
890 | + :return A python dictionary of the XML tree. |
891 | + """ |
892 | + tree_dict = {} |
893 | + |
894 | + root_dict = self.get_root_attributes() |
895 | + elements_dict = {} |
896 | + for element in self.get_elements_from_root(): |
897 | + elements_dict.update(self.element_to_dict(element)) |
898 | + |
899 | + if root_dict: |
900 | + tree_dict.update(root_dict) |
901 | + |
902 | + if elements_dict: |
903 | + if tree_dict: |
904 | + tree_dict.update(elements_dict) |
905 | + else: |
906 | + tree_dict = elements_dict |
907 | + |
908 | + return tree_dict |
909 | + |
910 | + |
911 | +class DictToXml(object): |
912 | + """ |
913 | + Class to convert a Python dictionary into an XML string . |
914 | + |
915 | + Simple usage: |
916 | + dict_2_xml = DictToXml(dictionary).dict_to_tree() |
917 | + """ |
918 | + |
919 | + def __init__(self, dictionary): |
920 | + """ |
921 | + Initialize the DictToXml class. |
922 | + |
923 | + :param dictionary: The dictionary to convert. |
924 | + :type dictionary dict |
925 | + """ |
926 | + if not isinstance(dictionary, dict): |
927 | + raise DictToXmlException("Type of parameter is not 'dict, " |
928 | + "got '%s' instead." % |
929 | + type(dictionary).__name__) |
930 | + super(DictToXml, self).__init__() |
931 | + self.dictionary = dictionary |
932 | + |
933 | + def dict_to_tree(self): |
934 | + """ |
935 | + Create an XML tree out of the provided dictionary. The XML structure |
936 | + is defined in the HACKING file. |
937 | + |
938 | + :return A string with the XML representation of the dictionary. |
939 | + """ |
940 | + dict_to_xml = XML_START |
941 | + |
942 | + loop = ET.Element(LOOP_ELEMENT) |
943 | + fields = ET.SubElement(loop, FIELDS_ELEMENT) |
944 | + for key, value in self.dictionary.iteritems(): |
945 | + # Get the type of the value. If it comes from Django, everything |
946 | + # will be unicode type, we default to 'str'. |
947 | + # TODO maybe have a mappings of fields<>type? |
948 | + value_type = type(value).__name__ |
949 | + if not value_type in VALID_TYPES: |
950 | + value_type = STR_TYPE |
951 | + attrib = {NAME_ATTR: key, TYPE_ATTR: value_type} |
952 | + field = ET.SubElement(fields, FIELD_ELEMENT, attrib) |
953 | + # ElementTree cannot serialize boolean types. |
954 | + field.text = str(value) |
955 | + |
956 | + # Dump the XML and add it to the valid start with its DTD. |
957 | + xml_dump = ET.tostring(loop) |
958 | + dict_to_xml += xml_dump |
959 | + |
960 | + return dict_to_xml |
Hey Milo great work !!1
Before discussing the dict necessity with danilo, I'll do a review here and lets land this, with tendency to refactor things later.
=== modified file 'dashboard/ frontend/ android_ textfield_ loop/tests/ test_android_ textfield_ loop_model. py' frontend/ android_ textfield_ loop/tests/ test_android_ textfield_ loop_model. py 2012-09-05 13:59:57 +0000 frontend/ android_ textfield_ loop/tests/ test_android_ textfield_ loop_model. py 2012-09-06 11:09:25 +0000
--- dashboard/
+++ dashboard/
@@ -24,9 +24,9 @@
A_NAME = 'a-build' VALID_LINES = ['a:2', 'b=3']
B_NAME = 'b-build'
- VALID_VALUES = 'a=2\nb=3'
+ VALID_VALUES = u'a=2\nb=3'
VALID_LINES = ['a=2', 'b=3']
- NON_VALID_VALUES = 'a:2\nb=3'
+ NON_VALID_VALUES = u'a:2\nb=3'
NON_
VALID_DICT = {'a': '2', 'b': '3'}
@@ -52,6 +52,7 @@ values_ wrong(self) :
self. assertEqual( (False, self.NON_ VALID_LINES) ,
AndroidTex tFieldLoop. valid_values(
self. NON_VALID_ VALUES) )
def test_valid_
+<<<<<<< TREE
def test_schedule_ build(self) : build_invalid( self): valid_android_ loop.schedule_ build()
self. assertEqual( build.result_ xml, {}) VALID_VALUES) ) build(self) : loop.schedule_ build() "UTF-8" ?>\n<!DOCTYPE ' \ ,fields) >\n' \ >\n]><loop> <fields> <field name="a">2</field>' \ >3</field> <field name="name" >a-build' \ restricted" >False< /field> ' \ official" >False< /field> ' \ >AndroidTextFie ldLoop< /field> ' \ l(expected_ out, build.result_xml) l(build. status, "success") build_invalid( self): valid_android_ loop.schedule_ build() l("", build.result_xml)
@@ -62,3 +63,27 @@
def test_schedule_
build = self.non_
+=======
+ self.NON_
+
+ def test_schedule_
+ build = self.android_
+ expected_out = '<?xml version="1.0" encoding=
+ 'loop [\n <!ELEMENT loop (description?
+ ' <!ELEMENT description (#PCDATA)>\n ' \
+ '<!ELEMENT fields (field+)>\n <!ELEMENT field ' \
+ '(#PCDATA)>\n <!ATTLIST field name CDATA ' \
+ '#REQUIRED>\n <!ATTLIST field type (text|int|bool) ' \
+ '"text"
+ '<field name="b"
+ '</field><field name="is_
+ '<field name="is_
+ '<field name="type"
+ '</fields></loop>'
+ self.assertEqua
+ self.assertEqua
+
+ def test_schedule_
+ build = self.non_
+ self.assertEqua
+>>>>>>> MERGE-SOURCE
What's with all the white spaces in the expected out? Can't we do strip() somewhere in the code and output xml without new lines and white spaces?
=== modified file 'dashboard/ frontend/ models/ loop_build. py' frontend/ models/ loop_build. py 2012-09-05 13:23:05 +0000 frontend/ models/ loop_build. py 2012-09-06 11:09:25 +0000
--- dashboard/
+++ dashboard/
@@ -18,6 +18,7 @@
from django.db import models models. loop import Loop lib.xml_ to_dict import XmlToDict
from frontend.
+from dashboard.
class LoopBuild( models. Model):
self. build_number = 1
@@ -47,3 +48,15 @@
else:
...