Merge lp:~zorba-coders/zorba/fixes_FOTS_driver_29_MAR into lp:zorba
- fixes_FOTS_driver_29_MAR
- Merge into trunk
Status: | Merged | ||||||||
---|---|---|---|---|---|---|---|---|---|
Approved by: | Chris Hillery | ||||||||
Approved revision: | 11332 | ||||||||
Merged at revision: | 11334 | ||||||||
Proposed branch: | lp:~zorba-coders/zorba/fixes_FOTS_driver_29_MAR | ||||||||
Merge into: | lp:zorba | ||||||||
Diff against target: |
308 lines (+100/-56) 7 files modified
test/fots/CMakeLists.txt (+0/-35) test/fots/ImportFOTS.cmake (+1/-1) test/fots_driver/FOTSZorbaManifest.xml (+3/-2) test/fots_driver/environment.xq (+20/-0) test/fots_driver/feedback.xq (+6/-2) test/fots_driver/fots-driver.xq (+14/-2) test/fots_driver/tools/dependencies.xq (+56/-14) |
||||||||
To merge this branch: | bzr merge lp:~zorba-coders/zorba/fixes_FOTS_driver_29_MAR | ||||||||
Related bugs: |
|
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Chris Hillery | Approve | ||
Sorin Marian Nasoi | Approve | ||
Review via email: mp+156109@code.launchpad.net |
Commit message
- updated FOTS snapshot to March 29th. Updated expected failures accordingly.
- if a test did not pass, we further check if the reason it failed is an environment dependency on setting of a default COLLECTION or COLLATION URI (other that Unicode Codepoint Collation). If so, we report the test case as "notRun" otherwise we report it as "fail". This change decreased the number of failing test cases by 34 (and increased the number of test cases "notRun" by the same amount).
- corrected errors in FOTSZorbaManife
- modified 'tools/
Description of the change
- updated FOTS snapshot to March 29th. Updated expected failures accordingly.
- if a test did not pass, we further check if the reason it failed is an environment dependency on setting of a default COLLECTION or COLLATION URI (other that Unicode Codepoint Collation). If so, we report the test case as "notRun" otherwise we report it as "fail". We first try running the test case because 130 test case actually PASS although their used environment requires setting a default COLECTION or COLLATION URI. This change decreased the number of failing test cases by 34 (and increased the number of test cases "notRun" by the same amount).
Sorin Marian Nasoi (sorin.marian.nasoi) : | # |
Zorba Build Bot (zorba-buildbot) wrote : | # |
Zorba Build Bot (zorba-buildbot) wrote : | # |
Validation queue job fixes_FOTS_
All tests succeeded!
Zorba Build Bot (zorba-buildbot) wrote : | # |
Voting does not meet specified criteria. Required: Approve > 1, Disapprove < 1, Needs Fixing < 1, Pending < 1, Needs Information < 1, Resubmit < 1. Got: 1 Approve.
Chris Hillery (ceejatec) wrote : | # |
The changes seem OK for now; however, they don't actually *fix* the two related bugs, do they?
Sorin Marian Nasoi (sorin.marian.nasoi) wrote : | # |
> The changes seem OK for now; however, they don't actually *fix* the two
> related bugs, do they?
They do actually because they correctly marked some bugs as notRun instead of fail (which according to W3C is OK).
If we decide we want to increase the number of test cases that PASS, additional work needs to be done in XQXQ for both of those bugs.
Sorin Marian Nasoi (sorin.marian.nasoi) wrote : | # |
Please also see:
https:/
My understanding of the comment is this:
Step 1) minimise the number of failing test cases as much as possible
Step 2) maximize the number of passing test cases as much as possible
What is your opinion?
Sorin Marian Nasoi (sorin.marian.nasoi) wrote : | # |
One more thing: even Mike Kay says that these 2 let's call them features (ability to define COLLECTIONS and to set COLLATION URI's) is like
"implicit dependency [...] An implementation that cannot set the context as required should behave in the same way as if a dependency has not been satisfied, ideally reporting the test as "notRun" with a reason."
taken from http://
So the proposed changed implement the ideal case where Zorba report's the test cases as "notRun" with information about the reason it did not run them.
Zorba Build Bot (zorba-buildbot) wrote : | # |
Validation queue starting for merge proposal.
Log at: http://
Zorba Build Bot (zorba-buildbot) wrote : | # |
The attempt to merge lp:~zorba-coders/zorba/fixes_FOTS_driver_29_MAR into lp:zorba failed. Below is the output from the failed tests.
CMake Error at /home/ceej/
Validation queue job fixes_FOTS_
finished. The final status was:
2 tests did not succeed - changes not commited.
Error in read script: /home/ceej/
Zorba Build Bot (zorba-buildbot) wrote : | # |
Validation queue starting for merge proposal.
Log at: http://
Zorba Build Bot (zorba-buildbot) wrote : | # |
Validation queue job fixes_FOTS_
All tests succeeded!
Preview Diff
1 | === modified file 'test/fots/CMakeLists.txt' | |||
2 | --- test/fots/CMakeLists.txt 2013-03-27 17:11:24 +0000 | |||
3 | +++ test/fots/CMakeLists.txt 2013-03-29 08:37:25 +0000 | |||
4 | @@ -135,31 +135,12 @@ | |||
5 | 135 | EXPECTED_FOTS_FAILURE (fn-available-environment-variables fn-available-environment-variables-011 0) | 135 | EXPECTED_FOTS_FAILURE (fn-available-environment-variables fn-available-environment-variables-011 0) |
6 | 136 | EXPECTED_FOTS_FAILURE (fn-avg cbcl-avg-002 0) | 136 | EXPECTED_FOTS_FAILURE (fn-avg cbcl-avg-002 0) |
7 | 137 | EXPECTED_FOTS_FAILURE (fn-avg cbcl-avg-003 0) | 137 | EXPECTED_FOTS_FAILURE (fn-avg cbcl-avg-003 0) |
8 | 138 | EXPECTED_FOTS_FAILURE (fn-collection fn-collection-4 0) | ||
9 | 139 | EXPECTED_FOTS_FAILURE (fn-collection fn-collection-4d 0) | ||
10 | 140 | EXPECTED_FOTS_FAILURE (fn-collection fn-collection-5 0) | ||
11 | 141 | EXPECTED_FOTS_FAILURE (fn-collection fn-collection-5d 0) | ||
12 | 142 | EXPECTED_FOTS_FAILURE (fn-collection fn-collection-6 0) | ||
13 | 143 | EXPECTED_FOTS_FAILURE (fn-collection fn-collection-8 0) | ||
14 | 144 | EXPECTED_FOTS_FAILURE (fn-collection fn-collection-9 0) | ||
15 | 145 | EXPECTED_FOTS_FAILURE (fn-collection fn-collection-10 0) | ||
16 | 146 | EXPECTED_FOTS_FAILURE (fn-collection fn-collection-10d 0) | ||
17 | 147 | EXPECTED_FOTS_FAILURE (fn-compare compare-010 0) | ||
18 | 148 | EXPECTED_FOTS_FAILURE (fn-deep-equal K-SeqDeepEqualFunc-56 0) | ||
19 | 149 | EXPECTED_FOTS_FAILURE (fn-deep-equal K-SeqDeepEqualFunc-57 0) | ||
20 | 150 | EXPECTED_FOTS_FAILURE (fn-deep-equal K-SeqDeepEqualFunc-58 0) | ||
21 | 151 | EXPECTED_FOTS_FAILURE (fn-deep-equal K-SeqDeepEqualFunc-59 0) | ||
22 | 152 | EXPECTED_FOTS_FAILURE (fn-deep-equal K-SeqDeepEqualFunc-60 0) | ||
23 | 153 | EXPECTED_FOTS_FAILURE (fn-deep-equal K-SeqDeepEqualFunc-61 0) | ||
24 | 154 | EXPECTED_FOTS_FAILURE (fn-deep-equal K-SeqDeepEqualFunc-62 0) | ||
25 | 155 | EXPECTED_FOTS_FAILURE (fn-deep-equal K-SeqDeepEqualFunc-63 0) | ||
26 | 156 | EXPECTED_FOTS_FAILURE (fn-deep-equal K2-SeqDeepEqualFunc-36 0) | 138 | EXPECTED_FOTS_FAILURE (fn-deep-equal K2-SeqDeepEqualFunc-36 0) |
27 | 157 | EXPECTED_FOTS_FAILURE (fn-deep-equal K2-SeqDeepEqualFunc-37 0) | 139 | EXPECTED_FOTS_FAILURE (fn-deep-equal K2-SeqDeepEqualFunc-37 0) |
28 | 158 | EXPECTED_FOTS_FAILURE (fn-deep-equal K2-SeqDeepEqualFunc-38 0) | 140 | EXPECTED_FOTS_FAILURE (fn-deep-equal K2-SeqDeepEqualFunc-38 0) |
29 | 159 | EXPECTED_FOTS_FAILURE (fn-deep-equal K2-SeqDeepEqualFunc-39 0) | 141 | EXPECTED_FOTS_FAILURE (fn-deep-equal K2-SeqDeepEqualFunc-39 0) |
30 | 160 | EXPECTED_FOTS_FAILURE (fn-distinct-values cbcl-distinct-values-002 0) | 142 | EXPECTED_FOTS_FAILURE (fn-distinct-values cbcl-distinct-values-002 0) |
31 | 161 | EXPECTED_FOTS_FAILURE (fn-distinct-values cbcl-distinct-values-002b 0) | 143 | EXPECTED_FOTS_FAILURE (fn-distinct-values cbcl-distinct-values-002b 0) |
32 | 162 | EXPECTED_FOTS_FAILURE (fn-distinct-values cbcl-distinct-values-003 0) | ||
33 | 163 | EXPECTED_FOTS_FAILURE (fn-environment-variable environment-variable-005 0) | 144 | EXPECTED_FOTS_FAILURE (fn-environment-variable environment-variable-005 0) |
34 | 164 | EXPECTED_FOTS_FAILURE (fn-environment-variable environment-variable-006 0) | 145 | EXPECTED_FOTS_FAILURE (fn-environment-variable environment-variable-006 0) |
35 | 165 | EXPECTED_FOTS_FAILURE (fn-environment-variable environment-variable-007 0) | 146 | EXPECTED_FOTS_FAILURE (fn-environment-variable environment-variable-007 0) |
36 | @@ -218,8 +199,6 @@ | |||
37 | 218 | EXPECTED_FOTS_FAILURE (fn-generate-id generate-id-004 0) | 199 | EXPECTED_FOTS_FAILURE (fn-generate-id generate-id-004 0) |
38 | 219 | EXPECTED_FOTS_FAILURE (fn-generate-id generate-id-005 0) | 200 | EXPECTED_FOTS_FAILURE (fn-generate-id generate-id-005 0) |
39 | 220 | EXPECTED_FOTS_FAILURE (fn-generate-id generate-id-006 0) | 201 | EXPECTED_FOTS_FAILURE (fn-generate-id generate-id-006 0) |
40 | 221 | EXPECTED_FOTS_FAILURE (fn-generate-id generate-id-012 0) | ||
41 | 222 | EXPECTED_FOTS_FAILURE (fn-generate-id generate-id-013 0) | ||
42 | 223 | EXPECTED_FOTS_FAILURE (fn-generate-id generate-id-014 0) | 202 | EXPECTED_FOTS_FAILURE (fn-generate-id generate-id-014 0) |
43 | 224 | EXPECTED_FOTS_FAILURE (fn-generate-id generate-id-015 0) | 203 | EXPECTED_FOTS_FAILURE (fn-generate-id generate-id-015 0) |
44 | 225 | EXPECTED_FOTS_FAILURE (fn-generate-id generate-id-017 0) | 204 | EXPECTED_FOTS_FAILURE (fn-generate-id generate-id-017 0) |
45 | @@ -285,8 +264,6 @@ | |||
46 | 285 | EXPECTED_FOTS_FAILURE (fn-subsequence cbcl-subsequence-017 0) | 264 | EXPECTED_FOTS_FAILURE (fn-subsequence cbcl-subsequence-017 0) |
47 | 286 | EXPECTED_FOTS_FAILURE (fn-subsequence cbcl-subsequence-018 0) | 265 | EXPECTED_FOTS_FAILURE (fn-subsequence cbcl-subsequence-018 0) |
48 | 287 | EXPECTED_FOTS_FAILURE (fn-subsequence cbcl-subsequence-024 0) | 266 | EXPECTED_FOTS_FAILURE (fn-subsequence cbcl-subsequence-024 0) |
49 | 288 | EXPECTED_FOTS_FAILURE (fn-substring-after fn-substring-after-24 0) | ||
50 | 289 | EXPECTED_FOTS_FAILURE (fn-substring-before fn-substring-before-24 0) | ||
51 | 290 | EXPECTED_FOTS_FAILURE (fn-tokenize fn-tokenize-31 0) | 267 | EXPECTED_FOTS_FAILURE (fn-tokenize fn-tokenize-31 0) |
52 | 291 | EXPECTED_FOTS_FAILURE (fn-tokenize fn-tokenize-32 0) | 268 | EXPECTED_FOTS_FAILURE (fn-tokenize fn-tokenize-32 0) |
53 | 292 | EXPECTED_FOTS_FAILURE (fn-unparsed-text fn-unparsed-text-037 0) | 269 | EXPECTED_FOTS_FAILURE (fn-unparsed-text fn-unparsed-text-037 0) |
54 | @@ -623,13 +600,7 @@ | |||
55 | 623 | EXPECTED_FOTS_FAILURE (xs-token cbcl-token-002b 0) | 600 | EXPECTED_FOTS_FAILURE (xs-token cbcl-token-002b 0) |
56 | 624 | EXPECTED_FOTS_FAILURE (xs-token cbcl-token-004 0) | 601 | EXPECTED_FOTS_FAILURE (xs-token cbcl-token-004 0) |
57 | 625 | EXPECTED_FOTS_FAILURE (fn-fold-left fold-left-009 0) | 602 | EXPECTED_FOTS_FAILURE (fn-fold-left fold-left-009 0) |
58 | 626 | EXPECTED_FOTS_FAILURE (fn-function-lookup fn-function-lookup-018 0) | ||
59 | 627 | EXPECTED_FOTS_FAILURE (fn-function-lookup fn-function-lookup-020 0) | ||
60 | 628 | EXPECTED_FOTS_FAILURE (fn-function-lookup fn-function-lookup-062 0) | 603 | EXPECTED_FOTS_FAILURE (fn-function-lookup fn-function-lookup-062 0) |
61 | 629 | EXPECTED_FOTS_FAILURE (fn-function-lookup fn-function-lookup-370 0) | ||
62 | 630 | EXPECTED_FOTS_FAILURE (fn-function-lookup fn-function-lookup-372 0) | ||
63 | 631 | EXPECTED_FOTS_FAILURE (fn-function-lookup fn-function-lookup-374 0) | ||
64 | 632 | EXPECTED_FOTS_FAILURE (fn-function-lookup fn-function-lookup-376 0) | ||
65 | 633 | EXPECTED_FOTS_FAILURE (fn-function-lookup fn-function-lookup-402 0) | 604 | EXPECTED_FOTS_FAILURE (fn-function-lookup fn-function-lookup-402 0) |
66 | 634 | EXPECTED_FOTS_FAILURE (fn-function-lookup fn-function-lookup-404 0) | 605 | EXPECTED_FOTS_FAILURE (fn-function-lookup fn-function-lookup-404 0) |
67 | 635 | EXPECTED_FOTS_FAILURE (fn-function-lookup fn-function-lookup-494 0) | 606 | EXPECTED_FOTS_FAILURE (fn-function-lookup fn-function-lookup-494 0) |
68 | @@ -658,13 +629,7 @@ | |||
69 | 658 | EXPECTED_FOTS_FAILURE (prod-FunctionCall FunctionCall-038 0) | 629 | EXPECTED_FOTS_FAILURE (prod-FunctionCall FunctionCall-038 0) |
70 | 659 | EXPECTED_FOTS_FAILURE (prod-FunctionCall FunctionCall-039 0) | 630 | EXPECTED_FOTS_FAILURE (prod-FunctionCall FunctionCall-039 0) |
71 | 660 | EXPECTED_FOTS_FAILURE (prod-InstanceofExpr instanceof140 0) | 631 | EXPECTED_FOTS_FAILURE (prod-InstanceofExpr instanceof140 0) |
72 | 661 | EXPECTED_FOTS_FAILURE (prod-NamedFunctionRef function-literal-018 0) | ||
73 | 662 | EXPECTED_FOTS_FAILURE (prod-NamedFunctionRef function-literal-020 0) | ||
74 | 663 | EXPECTED_FOTS_FAILURE (prod-NamedFunctionRef function-literal-062 0) | 632 | EXPECTED_FOTS_FAILURE (prod-NamedFunctionRef function-literal-062 0) |
75 | 664 | EXPECTED_FOTS_FAILURE (prod-NamedFunctionRef function-literal-370 0) | ||
76 | 665 | EXPECTED_FOTS_FAILURE (prod-NamedFunctionRef function-literal-372 0) | ||
77 | 666 | EXPECTED_FOTS_FAILURE (prod-NamedFunctionRef function-literal-374 0) | ||
78 | 667 | EXPECTED_FOTS_FAILURE (prod-NamedFunctionRef function-literal-376 0) | ||
79 | 668 | EXPECTED_FOTS_FAILURE (prod-NamedFunctionRef function-literal-402 0) | 633 | EXPECTED_FOTS_FAILURE (prod-NamedFunctionRef function-literal-402 0) |
80 | 669 | EXPECTED_FOTS_FAILURE (prod-NamedFunctionRef function-literal-404 0) | 634 | EXPECTED_FOTS_FAILURE (prod-NamedFunctionRef function-literal-404 0) |
81 | 670 | EXPECTED_FOTS_FAILURE (prod-NamedFunctionRef function-literal-494 0) | 635 | EXPECTED_FOTS_FAILURE (prod-NamedFunctionRef function-literal-494 0) |
82 | 671 | 636 | ||
83 | === modified file 'test/fots/ImportFOTS.cmake' | |||
84 | --- test/fots/ImportFOTS.cmake 2013-03-21 20:02:48 +0000 | |||
85 | +++ test/fots/ImportFOTS.cmake 2013-03-29 08:37:25 +0000 | |||
86 | @@ -19,7 +19,7 @@ | |||
87 | 19 | 19 | ||
88 | 20 | 20 | ||
89 | 21 | # Change this to publish updated FOTS archives | 21 | # Change this to publish updated FOTS archives |
91 | 22 | SET (FOTS_ARCHIVE "FOTS_210313.tgz") | 22 | SET (FOTS_ARCHIVE "FOTS_290313.tgz") |
92 | 23 | 23 | ||
93 | 24 | # Change this to modify which elements in FOTS driver results are output | 24 | # Change this to modify which elements in FOTS driver results are output |
94 | 25 | # as CDATA | 25 | # as CDATA |
95 | 26 | 26 | ||
96 | === modified file 'test/fots_driver/FOTSZorbaManifest.xml' | |||
97 | --- test/fots_driver/FOTSZorbaManifest.xml 2013-03-15 08:22:41 +0000 | |||
98 | +++ test/fots_driver/FOTSZorbaManifest.xml 2013-03-29 08:37:25 +0000 | |||
99 | @@ -1,6 +1,6 @@ | |||
100 | 1 | <test-suite-result xmlns="http://www.w3.org/2010/09/qt-fots-catalog"> | 1 | <test-suite-result xmlns="http://www.w3.org/2010/09/qt-fots-catalog"> |
103 | 2 | <dependency type="feature" value="collection-stability" satisfied="true"/> <!-- Is this correct? --> | 2 | <dependency type="feature" value="collection-stability" satisfied="false"/> |
104 | 3 | <dependency type="feature" value="directory-as-collection-uri" satisfied="true"/> <!-- Is this correct? --> | 3 | <dependency type="feature" value="directory-as-collection-uri" satisfied="false"/> |
105 | 4 | <dependency type="feature" value="higherOrderFunctions" satisfied="true"/> | 4 | <dependency type="feature" value="higherOrderFunctions" satisfied="true"/> |
106 | 5 | <dependency type="feature" value="moduleImport" satisfied="true"/> | 5 | <dependency type="feature" value="moduleImport" satisfied="true"/> |
107 | 6 | <dependency type="feature" value="namespace-axis" satisfied="false"/> <!-- Is this correct? --> | 6 | <dependency type="feature" value="namespace-axis" satisfied="false"/> <!-- Is this correct? --> |
108 | @@ -14,6 +14,7 @@ | |||
109 | 14 | <dependency type="spec" value="XP10 XQ10" satisfied="true"/> | 14 | <dependency type="spec" value="XP10 XQ10" satisfied="true"/> |
110 | 15 | <dependency type="spec" value="XP10+" satisfied="true"/> | 15 | <dependency type="spec" value="XP10+" satisfied="true"/> |
111 | 16 | <dependency type="spec" value="XP20 XQ10" satisfied="true"/> | 16 | <dependency type="spec" value="XP20 XQ10" satisfied="true"/> |
112 | 17 | <dependency type="spec" value="XP20" satisfied="true"/> | ||
113 | 17 | <dependency type="spec" value="XP20+" satisfied="true"/> | 18 | <dependency type="spec" value="XP20+" satisfied="true"/> |
114 | 18 | <dependency type="spec" value="XP30+" satisfied="true"/> | 19 | <dependency type="spec" value="XP30+" satisfied="true"/> |
115 | 19 | <dependency type="spec" value="XP30+ XQ10+" satisfied="true"/> | 20 | <dependency type="spec" value="XP30+ XQ10+" satisfied="true"/> |
116 | 20 | 21 | ||
117 | === modified file 'test/fots_driver/environment.xq' | |||
118 | --- test/fots_driver/environment.xq 2013-03-26 16:25:16 +0000 | |||
119 | +++ test/fots_driver/environment.xq 2013-03-29 08:37:25 +0000 | |||
120 | @@ -759,3 +759,23 @@ | |||
121 | 759 | else () | 759 | else () |
122 | 760 | } | 760 | } |
123 | 761 | }; | 761 | }; |
124 | 762 | |||
125 | 763 | (:~ | ||
126 | 764 | : Checks if a test case should be run. Possible reasons for not running a test | ||
127 | 765 | : according to http://dev.w3.org/2011/QT3-test-suite/guide/running.html: | ||
128 | 766 | : - the environment requires the setting of collections | ||
129 | 767 | : - the environment requires the setting of collation URIs | ||
130 | 768 | : | ||
131 | 769 | : | ||
132 | 770 | :) | ||
133 | 771 | declare function env:check-prerequisites( | ||
134 | 772 | $case as element(fots:test-case), | ||
135 | 773 | $env as element(fots:environment)? | ||
136 | 774 | ) as xs:string? | ||
137 | 775 | { | ||
138 | 776 | if(exists($case/fots:environment/fots:collection) or exists($env/fots:collection)) | ||
139 | 777 | then 'Default collection is always an empty sequence.' | ||
140 | 778 | else if (exists($case/fots:environment/fots:collation) or exists($env/fots:collation)) | ||
141 | 779 | then 'Can not define any other collations (other than the Unicode Codepoint Collation).' | ||
142 | 780 | else () | ||
143 | 781 | }; | ||
144 | 762 | 782 | ||
145 | === modified file 'test/fots_driver/feedback.xq' | |||
146 | --- test/fots_driver/feedback.xq 2013-03-18 12:53:46 +0000 | |||
147 | +++ test/fots_driver/feedback.xq 2013-03-29 08:37:25 +0000 | |||
148 | @@ -211,16 +211,20 @@ | |||
149 | 211 | : in the $exceptedTestCases global var). | 211 | : in the $exceptedTestCases global var). |
150 | 212 | : | 212 | : |
151 | 213 | : @param $case test case. | 213 | : @param $case test case. |
152 | 214 | : @param $error the reason for which the test case was not run. | ||
153 | 214 | : @return the test case. | 215 | : @return the test case. |
154 | 215 | :) | 216 | :) |
155 | 216 | declare function feedback:not-run( | 217 | declare function feedback:not-run( |
157 | 217 | $case as element(fots:test-case) | 218 | $case as element(fots:test-case), |
158 | 219 | $error as xs:string? | ||
159 | 218 | ) as element(fots:test-case)? | 220 | ) as element(fots:test-case)? |
160 | 219 | { | 221 | { |
161 | 220 | trace(data($case/@name), "processing test case :"); | 222 | trace(data($case/@name), "processing test case :"); |
162 | 221 | trace("Above test case was not run.",""); | 223 | trace("Above test case was not run.",""); |
163 | 222 | 224 | ||
165 | 223 | <fots:test-case name="{$case/@name}" result="notRun" /> | 225 | if(exists($error)) |
166 | 226 | then <fots:test-case name="{$case/@name}" result="notRun" comment="{$error}"/> | ||
167 | 227 | else <fots:test-case name="{$case/@name}" result="notRun" /> | ||
168 | 224 | }; | 228 | }; |
169 | 225 | 229 | ||
170 | 226 | 230 | ||
171 | 227 | 231 | ||
172 | === modified file 'test/fots_driver/fots-driver.xq' | |||
173 | --- test/fots_driver/fots-driver.xq 2013-03-21 18:43:40 +0000 | |||
174 | +++ test/fots_driver/fots-driver.xq 2013-03-29 08:37:25 +0000 | |||
175 | @@ -667,7 +667,7 @@ | |||
176 | 667 | return | 667 | return |
177 | 668 | if ($isExcepted) | 668 | if ($isExcepted) |
178 | 669 | then | 669 | then |
180 | 670 | feedback:not-run($testCase) | 670 | feedback:not-run($testCase, ()) |
181 | 671 | else | 671 | else |
182 | 672 | feedback:not-applicable($testCase, | 672 | feedback:not-applicable($testCase, |
183 | 673 | string-join($depMet,'')) | 673 | string-join($depMet,'')) |
184 | @@ -711,7 +711,7 @@ | |||
185 | 711 | $verbose, | 711 | $verbose, |
186 | 712 | fn:false()) | 712 | fn:false()) |
187 | 713 | else | 713 | else |
189 | 714 | feedback:not-run($testCase) | 714 | feedback:not-run($testCase,()) |
190 | 715 | } | 715 | } |
191 | 716 | else if (exists($depMet)) then | 716 | else if (exists($depMet)) then |
192 | 717 | { | 717 | { |
193 | @@ -859,6 +859,8 @@ | |||
194 | 859 | $testSetBaseURI); | 859 | $testSetBaseURI); |
195 | 860 | 860 | ||
196 | 861 | variable $duration := (datetime:current-dateTime() - $startDateTime); | 861 | variable $duration := (datetime:current-dateTime() - $startDateTime); |
197 | 862 | |||
198 | 863 | variable $prerequisitesError as xs:string? := env:check-prerequisites($case, $env); | ||
199 | 862 | 864 | ||
200 | 863 | if (feedback:check-pass($result, $queryName, $testSetName, $expectedFailure)) | 865 | if (feedback:check-pass($result, $queryName, $testSetName, $expectedFailure)) |
201 | 864 | then | 866 | then |
202 | @@ -869,6 +871,16 @@ | |||
203 | 869 | $duration, | 871 | $duration, |
204 | 870 | $verbose, | 872 | $verbose, |
205 | 871 | exists($expectedFailure)) | 873 | exists($expectedFailure)) |
206 | 874 | (: | ||
207 | 875 | If the test case did not pass, we check to see if the failure is caused | ||
208 | 876 | by a environment that requires setting of a COLLATION or COLLECTION. | ||
209 | 877 | There are over 130 test cases that are using an environment that requires | ||
210 | 878 | setting a COLLATION or COLLECTION but they still PASS even if this setting | ||
211 | 879 | is not done. That is why we first run the test case. | ||
212 | 880 | :) | ||
213 | 881 | else if(exists($prerequisitesError)) | ||
214 | 882 | then | ||
215 | 883 | feedback:not-run($case, $prerequisitesError) | ||
216 | 872 | else | 884 | else |
217 | 873 | feedback:fail($case, | 885 | feedback:fail($case, |
218 | 874 | $result, | 886 | $result, |
219 | 875 | 887 | ||
220 | === modified file 'test/fots_driver/tools/dependencies.xq' | |||
221 | --- test/fots_driver/tools/dependencies.xq 2012-11-13 10:45:24 +0000 | |||
222 | +++ test/fots_driver/tools/dependencies.xq 2013-03-29 08:37:25 +0000 | |||
223 | @@ -1,27 +1,69 @@ | |||
225 | 1 | (: this query loads a rezults.xml file and reports the number of 'non applicable' tests grouped by feature :) | 1 | (: this query loads a results.xml file and reports the number of 'non applicable' tests grouped by feature :) |
226 | 2 | (: | 2 | (: |
228 | 3 | sample usage | 3 | sample usage (please note that base-uri has to be an absolute URI) |
229 | 4 | |||
230 | 4 | ./zorba -f -q ../../test/fots_driver/tools/dependencies.xq | 5 | ./zorba -f -q ../../test/fots_driver/tools/dependencies.xq |
232 | 5 | -e pathFailures:=results_12_nov.xml | 6 | -e pathFailures:=results.xml |
233 | 7 | --base-uri "/home/spungi/work/zorba/repo/tmp/fix_fots_driver/build/test/fots/2011/QT3-test-suite/catalog.xml" | ||
234 | 6 | -o driver_dependencies.xml | 8 | -o driver_dependencies.xml |
235 | 7 | --indent | 9 | --indent |
237 | 8 | :) | 10 | :) |
238 | 9 | 11 | ||
239 | 10 | import module namespace file = "http://expath.org/ns/file"; | 12 | import module namespace file = "http://expath.org/ns/file"; |
240 | 13 | import schema namespace fots = "http://www.w3.org/2010/09/qt-fots-catalog" at "catalog-schema.xsd"; | ||
241 | 11 | 14 | ||
242 | 12 | declare default element namespace "http://www.w3.org/2010/09/qt-fots-catalog"; | 15 | declare default element namespace "http://www.w3.org/2010/09/qt-fots-catalog"; |
243 | 13 | declare namespace fots = "http://www.w3.org/2010/09/qt-fots-catalog"; | ||
244 | 14 | 16 | ||
245 | 15 | declare variable $pathFailures external; | 17 | declare variable $pathFailures external; |
247 | 16 | 18 | declare variable $pathFOTSZorbaManifest external := | |
248 | 19 | "../../../../../test/fots_driver/FOTSZorbaManifest.xml"; | ||
249 | 20 | |||
250 | 21 | (:~ | ||
251 | 22 | : This function is actually written by Michael Kay, Saxonica 18 July 2011 and | ||
252 | 23 | : is part of the http://dev.w3.org/cvsweb/2011/QT3-test-suite/tools/dependencies.xq | ||
253 | 24 | :) | ||
254 | 25 | declare function local:mk_dependencies($t as element(fots:test-case)) as element(fots:dependency)* { | ||
255 | 26 | $t/fots:dependency | $t/../fots:dependency | ||
256 | 27 | }; | ||
257 | 28 | |||
258 | 29 | (:~ | ||
259 | 30 | : This function is actually written by Michael Kay, Saxonica 18 July 2011 and | ||
260 | 31 | : is part of the http://dev.w3.org/cvsweb/2011/QT3-test-suite/tools/dependencies.xq | ||
261 | 32 | :) | ||
262 | 33 | declare function local:mk_FOTS_dependencies() as element(fots:dependency)* { | ||
263 | 34 | let $test-sets := | ||
264 | 35 | for $s in doc('catalog.xml')/fots:catalog/fots:test-set | ||
265 | 36 | let $uri := resolve-uri($s/@file, base-uri($s)) | ||
266 | 37 | return doc($uri)/* | ||
267 | 38 | for $d in distinct-values($test-sets//fots:dependency/@type) | ||
268 | 39 | return | ||
269 | 40 | <dependency type="{$d}"> { | ||
270 | 41 | let $de := $test-sets//fots:dependency[@type = $d] | ||
271 | 42 | for $v in distinct-values($de/@value) | ||
272 | 43 | return | ||
273 | 44 | <value number-of-tests="{count($test-sets/fots:test-case[local:mk_dependencies(.)[@type = $d and @value = $v]])}"> | ||
274 | 45 | {$v} | ||
275 | 46 | </value> | ||
276 | 47 | }</dependency> | ||
277 | 48 | }; | ||
278 | 49 | |||
279 | 17 | variable $failures := fn:parse-xml(file:read-text($pathFailures)); | 50 | variable $failures := fn:parse-xml(file:read-text($pathFailures)); |
280 | 51 | variable $FOTSZorbaManifest := doc(resolve-uri($pathFOTSZorbaManifest)); | ||
281 | 52 | variable $notDefinedInZorba := | ||
282 | 53 | for $dep in local:mk_FOTS_dependencies() | ||
283 | 54 | for $depval in $dep/value | ||
284 | 55 | where empty($FOTSZorbaManifest/fots:test-suite-result/fots:dependency[@type=$dep/@type and @value=$depval]) | ||
285 | 56 | return <dependency type="{$dep/@type}" value="{$depval/text()}" />; | ||
286 | 18 | 57 | ||
290 | 19 | <report> { | 58 | <report number-of-tests="{count($failures/fots:test-cases/fots:test-set/fots:test-case[@result="n/a"])}" > { |
291 | 20 | for $d in distinct-values(data($failures/fots:test-cases/fots:test-set/fots:test-case[@result="not applicable"]/@comment)) | 59 | (for $d in distinct-values(data($failures/fots:test-cases/fots:test-set/fots:test-case[@result="n/a"]/@comment)) |
292 | 21 | order by $d | 60 | let $failures := count($failures/fots:test-cases//fots:test-set//fots:test-case[@result="n/a" and @comment=$d]) |
293 | 61 | order by $failures descending, $d | ||
294 | 22 | return | 62 | return |
295 | 23 | <dependency type="{$d}"> { | ||
296 | 24 | <value number-of-tests="{count($failures/fots:test-cases//fots:test-set//fots:test-case[@result="not applicable" and @comment=$d])}" /> | ||
297 | 25 | }</dependency> | ||
298 | 26 | }</report> | ||
299 | 27 | |||
300 | 28 | \ No newline at end of file | 63 | \ No newline at end of file |
301 | 64 | <dependency number-of-tests="{$failures}" type="{$d}" />, | ||
302 | 65 | if (empty($notDefinedInZorba)) | ||
303 | 66 | then <!-- There are no dependencies in FOTS that are not declared in FOTSZorbaManifest.xml --> | ||
304 | 67 | else (<!-- Following dependency from FOTS is not defined in FOTSZorbaManifest.xml -->, | ||
305 | 68 | $notDefinedInZorba) | ||
306 | 69 | ) | ||
307 | 70 | }</report> | ||
308 | 29 | \ No newline at end of file | 71 | \ No newline at end of file |
Validation queue starting for merge proposal. zorbatest. lambda. nu:8080/ remotequeue/ fixes_FOTS_ driver_ 29_MAR- 2013-03- 29T06-29- 40.76Z/ log.html
Log at: http://