Merge lp:~fgimenez/snapcraft/build-examples-test into lp:~snappy-dev/snapcraft/core
- build-examples-test
- Merge into core
Status: | Merged |
---|---|
Approved by: | Leo Arias |
Approved revision: | 178 |
Merged at revision: | 197 |
Proposed branch: | lp:~fgimenez/snapcraft/build-examples-test |
Merge into: | lp:~snappy-dev/snapcraft/core |
Diff against target: |
318 lines (+153/-61) 13 files modified
debian/control (+11/-0) debian/rules (+3/-0) debian/snapcraft-examples.install (+1/-0) debian/snapcraft.install (+3/-0) debian/source/options (+0/-5) debian/tests/control (+3/-2) debian/tests/runexamples (+2/-0) debian/tests/runtests (+2/-0) examples/downloader-with-wiki-parts/snapcraft.yaml (+1/-0) integration-tests/runtests.sh (+44/-18) integration-tests/units/examples.pxu (+10/-0) integration-tests/units/testplans.pxu (+6/-1) runtests.sh (+67/-35) |
To merge this branch: | bzr merge lp:~fgimenez/snapcraft/build-examples-test |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Leo Arias (community) | Needs Fixing | ||
Daniel Holbach (community) | Approve | ||
Sergio Schvezov | Needs Fixing | ||
Review via email: mp+270798@code.launchpad.net |
Commit message
plainbox test for building the examples
Description of the change
plainbox test for building the examples
Leo Arias (elopio) wrote : | # |
I agree with what sergio says. I'd also like to see snapcraft run on each example, and check that it does what it's supposed to. That's way more complex and I have no idea yet how to do it, so +1 to this branch that gets all started.
fwiw, I did a little refactor in here to support dep8:
https:/
You could put the script inside examples dir.
Thanks Federico.
Federico Gimenez (fgimenez) wrote : | # |
Thanks a lot Sergio and Leo for your comments.
With the latest changes it can be invoked with the options suggested by Sergio, by default it only runs the normal test plan in plainbox, to run the examples test plan you have to be explicit:
./runtests.sh plainbox examples
Let me know what do you think. Leo, your dep8 work is already merged, right?
Daniel Holbach (dholbach) wrote : | # |
I think the following changes would be good: http://
Unfortunately does the runexamples test still explode: http://
Daniel Holbach (dholbach) wrote : | # |
And yep, the dep8 branch is in trunk.
Federico Gimenez (fgimenez) wrote : | # |
Back to wip, autopkgtest not yet working
Federico Gimenez (fgimenez) wrote : | # |
It should be fixed now, the examples folder was not being added to the deb and because of that was not available in the testbed, let me know if it works for you.
- 172. By Federico Gimenez
-
added missing $
Sergio Schvezov (sergiusens) wrote : | # |
As discussed during the standup, can we have a snapcraft-examples binary package that depends on the snapcraft binary package?
Leo Arias (elopio) wrote : | # |
+Tests: runtests
+ runexamples
I would call them integration-tests and example-tests, or something like that.
I ran the tests, and they work. Got one failure, which is awesome. Thanks Federico!
Waiting for the examples package to approve.
Daniel Holbach (dholbach) wrote : | # |
- 173. By Federico Gimenez
-
merged trunk
- 174. By Federico Gimenez
- 175. By Federico Gimenez
-
fixed paths to work with the snappy-examples package
- 176. By Federico Gimenez
-
merged trunk
- 177. By Federico Gimenez
-
Removed libssl-dev dependency
Daniel Holbach (dholbach) wrote : | # |
Can you apply the following change?
== modified file 'examples/
--- examples/
+++ examples/
@@ -14,3 +14,4 @@
source: .
after:
- curl
+build-packages: [libssl-dev]
- 178. By Federico Gimenez
-
added build dependency on libssl-dev to downloader-
withwiki- parts
Federico Gimenez (fgimenez) wrote : | # |
@Daniel, thanks, done :)
Daniel Holbach (dholbach) wrote : | # |
Thanks a lot! Good work!
Leo Arias (elopio) wrote : | # |
There is one lintian error. Fixed with:
http://
Leo Arias (elopio) wrote : | # |
Lets land this one, the lintian fix is here: https:/
Preview Diff
1 | === modified file 'debian/control' |
2 | --- debian/control 2015-09-16 14:26:10 +0000 |
3 | +++ debian/control 2015-09-22 08:24:43 +0000 |
4 | @@ -39,3 +39,14 @@ |
5 | a single toolset, but instead is a collection of tools that enable the |
6 | natural workflow of an upstream to be extended with a simple release step |
7 | into Snappy. |
8 | + |
9 | +Package: snapcraft-examples |
10 | +Architecture: all |
11 | +Depends: snapcraft (= ${binary:Version}), ${misc:Depends}, ${python:Depends} |
12 | +Description: examples for snapcraft |
13 | + Snapcraft aims to make upstream developers' lives easier and as such is not |
14 | + a single toolset, but instead is a collection of tools that enable the |
15 | + natural workflow of an upstream to be extended with a simple release step |
16 | + into Snappy. |
17 | + . |
18 | + This package contains examples for snapcraft. |
19 | |
20 | === modified file 'debian/rules' |
21 | --- debian/rules 2015-09-16 14:26:10 +0000 |
22 | +++ debian/rules 2015-09-22 08:24:43 +0000 |
23 | @@ -3,3 +3,6 @@ |
24 | |
25 | %: |
26 | dh $@ --with python3 --buildsystem=pybuild |
27 | + |
28 | +override_dh_compress: |
29 | + dh_compress -X/usr/share/snapcraft/examples |
30 | |
31 | === added file 'debian/snapcraft-examples.install' |
32 | --- debian/snapcraft-examples.install 1970-01-01 00:00:00 +0000 |
33 | +++ debian/snapcraft-examples.install 2015-09-22 08:24:43 +0000 |
34 | @@ -0,0 +1,1 @@ |
35 | +examples /usr/share/snapcraft |
36 | |
37 | === added file 'debian/snapcraft.install' |
38 | --- debian/snapcraft.install 1970-01-01 00:00:00 +0000 |
39 | +++ debian/snapcraft.install 2015-09-22 08:24:43 +0000 |
40 | @@ -0,0 +1,3 @@ |
41 | +/usr/bin |
42 | +/usr/lib/python* |
43 | +/usr/share/snapcraft |
44 | |
45 | === modified file 'debian/source/options' |
46 | --- debian/source/options 2015-07-06 16:48:22 +0000 |
47 | +++ debian/source/options 2015-09-22 08:24:43 +0000 |
48 | @@ -1,7 +1,2 @@ |
49 | # Ignore standard files (like .bzr) |
50 | tar-ignore |
51 | - |
52 | -# Ignore examples folder because it likely contains a bunch of stuff we don't |
53 | -# care about (like downloaded go1.4 binaries and such). It's not important |
54 | -# for it to be distributed with the source. |
55 | -tar-ignore = "examples" |
56 | |
57 | === modified file 'debian/tests/control' |
58 | --- debian/tests/control 2015-09-17 14:56:20 +0000 |
59 | +++ debian/tests/control 2015-09-22 08:24:43 +0000 |
60 | @@ -1,5 +1,6 @@ |
61 | -Test-Command: ./runtests.sh |
62 | -Restrictions: allow-stderr, isolation-container |
63 | +Tests: runtests |
64 | + runexamples |
65 | +Restrictions: allow-stderr, isolation-container, rw-build-tree |
66 | Depends: @, |
67 | build-essential, |
68 | pep8, |
69 | |
70 | === added file 'debian/tests/runexamples' |
71 | --- debian/tests/runexamples 1970-01-01 00:00:00 +0000 |
72 | +++ debian/tests/runexamples 2015-09-22 08:24:43 +0000 |
73 | @@ -0,0 +1,2 @@ |
74 | +#!/bin/sh |
75 | +./runtests.sh plainbox examples |
76 | |
77 | === added file 'debian/tests/runtests' |
78 | --- debian/tests/runtests 1970-01-01 00:00:00 +0000 |
79 | +++ debian/tests/runtests 2015-09-22 08:24:43 +0000 |
80 | @@ -0,0 +1,2 @@ |
81 | +#!/bin/sh |
82 | +./runtests.sh |
83 | |
84 | === modified file 'examples/downloader-with-wiki-parts/snapcraft.yaml' |
85 | --- examples/downloader-with-wiki-parts/snapcraft.yaml 2015-09-21 05:04:31 +0000 |
86 | +++ examples/downloader-with-wiki-parts/snapcraft.yaml 2015-09-22 08:24:43 +0000 |
87 | @@ -14,3 +14,4 @@ |
88 | source: . |
89 | after: |
90 | - curl |
91 | +build-packages: [libssl-dev] |
92 | |
93 | === modified file 'integration-tests/runtests.sh' |
94 | --- integration-tests/runtests.sh 2015-08-04 23:46:54 +0000 |
95 | +++ integration-tests/runtests.sh 2015-09-22 08:24:43 +0000 |
96 | @@ -17,24 +17,45 @@ |
97 | |
98 | set -e |
99 | |
100 | -if [ -z "$SNAPCRAFT" ]; then |
101 | - export SNAPCRAFT=snapcraft |
102 | -fi |
103 | - |
104 | -# Create a temporary directory so that we can run 'manage.py develop' and |
105 | -# create the .provider file there |
106 | -temp_dir=$(mktemp -d) |
107 | -# Develop the provider, this will let us run tests on it |
108 | -./manage.py develop -d $temp_dir |
109 | -# Set PROVIDERPATH (see plainbox(1)) so that we can see the provider |
110 | -# without installing it. |
111 | -export PROVIDERPATH=$PROVIDERPATH:$temp_dir |
112 | -# Run the 'normal' test plan |
113 | -plainbox run \ |
114 | - -T 2015.com.canonical.snapcraft::normal \ |
115 | - -f json -o $temp_dir/result.json |
116 | -# Analyze the result and fail if there are any failures |
117 | -python3 - << __PYTHON__ |
118 | +parseargs(){ |
119 | + export TEST_PLAN=$1 |
120 | +} |
121 | + |
122 | +run_test_plan(){ |
123 | + if [ -z "$SNAPCRAFT" ]; then |
124 | + export SNAPCRAFT=snapcraft |
125 | + fi |
126 | + |
127 | + # Create a temporary directory so that we can run 'manage.py develop' and |
128 | + # create the .provider file there |
129 | + temp_dir=$(mktemp -d) |
130 | + # Develop the provider, this will let us run tests on it |
131 | + ./manage.py develop -d $temp_dir |
132 | + # Set PROVIDERPATH (see plainbox(1)) so that we can see the provider |
133 | + # without installing it. |
134 | + export PROVIDERPATH=$PROVIDERPATH:$temp_dir |
135 | + # create symlink from the provider's data directory to examples for them to |
136 | + # be available to the tests |
137 | + if [ $TEST_PLAN == "examples" ]; then |
138 | + BASEDIR=$( cd "$( dirname "${BASH_SOURCE[0]}" )"/.. && pwd ) |
139 | + TARGET_PATH=${BASEDIR}/integration-tests/data/examples |
140 | + if [ -z $ADT_ARTIFACTS ]; then |
141 | + EXAMPLES_PATH=${BASEDIR}/examples |
142 | + else |
143 | + EXAMPLES_PATH=/usr/share/snapcraft/examples |
144 | + fi |
145 | + ln -fs ${EXAMPLES_PATH} ${TARGET_PATH} |
146 | + fi |
147 | + # Run the test plan |
148 | + plainbox run \ |
149 | + -T 2015.com.canonical.snapcraft::"$TEST_PLAN" \ |
150 | + -f json -o $temp_dir/result.json |
151 | + # remove the examples symlink |
152 | + if [ $TEST_PLAN == "examples" ]; then |
153 | + rm ${TARGET_PATH} |
154 | + fi |
155 | + # Analyze the result and fail if there are any failures |
156 | + python3 - << __PYTHON__ |
157 | import json |
158 | with open("$temp_dir/result.json", "rt", encoding="utf-8") as stream: |
159 | results = json.load(stream) |
160 | @@ -46,3 +67,8 @@ |
161 | print("Overall: {0}".format("fail" if failed else "pass")) |
162 | raise SystemExit(failed) |
163 | __PYTHON__ |
164 | +} |
165 | + |
166 | +parseargs "$@" |
167 | + |
168 | +run_test_plan |
169 | |
170 | === added file 'integration-tests/units/examples.pxu' |
171 | --- integration-tests/units/examples.pxu 1970-01-01 00:00:00 +0000 |
172 | +++ integration-tests/units/examples.pxu 2015-09-22 08:24:43 +0000 |
173 | @@ -0,0 +1,10 @@ |
174 | +id: snapcraft/examples/build |
175 | +plugin: shell |
176 | +command: |
177 | + set -e |
178 | + for example in ${PLAINBOX_PROVIDER_DATA}/examples/* |
179 | + do |
180 | + cp -rT ${example//:} ${PLAINBOX_SESSION_SHARE} |
181 | + cd ${PLAINBOX_SESSION_SHARE} && ${SNAPCRAFT} |
182 | + rm -rf ${PLAINBOX_SESSION_SHARE}/* |
183 | + done |
184 | |
185 | === modified file 'integration-tests/units/testplans.pxu' |
186 | --- integration-tests/units/testplans.pxu 2015-07-02 18:29:47 +0000 |
187 | +++ integration-tests/units/testplans.pxu 2015-09-22 08:24:43 +0000 |
188 | @@ -1,4 +1,9 @@ |
189 | unit: test plan |
190 | id: normal |
191 | -name: Tests for Snapcraft |
192 | +name: Tests for Snapcraft |
193 | include: snapcraft/normal/.* |
194 | + |
195 | +unit: test plan |
196 | +id: examples |
197 | +name: Test for building Snapcraft examples |
198 | +include: snapcraft/examples/.* |
199 | |
200 | === modified file 'runtests.sh' |
201 | --- runtests.sh 2015-09-17 07:31:24 +0000 |
202 | +++ runtests.sh 2015-09-22 08:24:43 +0000 |
203 | @@ -20,39 +20,62 @@ |
204 | export PATH=$(pwd)/bin:$PATH |
205 | export PYTHONPATH=$(pwd):$PYTHONPATH |
206 | |
207 | -SRC_PATHS="bin snapcraft snapcraft/tests" |
208 | - |
209 | -# These three checks could easily be done with flake8 in one shot if |
210 | -# we had python3-flake8 provide flake8 |
211 | -# Ignore 501 (line-too-long) |
212 | -pep8 $SRC_PATHS --ignore=E501 |
213 | - |
214 | -pyflakes3 $SRC_PATHS |
215 | - |
216 | -# mccabe in 'warning' mode as we have high complexity |
217 | -mccabe_list= |
218 | -for unit in $(find snapcraft -type f -name '*.py') |
219 | -do |
220 | - output=$(python3 -m mccabe --min 10 "$unit") |
221 | - [ -n "$output" ] && mccabe_list="- $unit:\n $output\n$mccabe_list" |
222 | -done |
223 | - |
224 | -if [ -n "$mccabe_list" ]; then |
225 | - echo -e "\e[1;31mThe project has gotten complex\e[0m." |
226 | - echo "Here's the list of units exceeding 10:" |
227 | - echo -e "$mccabe_list" |
228 | -fi |
229 | - |
230 | -if which python3-coverage >/dev/null 2>&1; then |
231 | - python3-coverage erase |
232 | - python3-coverage run --branch --source snapcraft -m unittest |
233 | - mv .coverage .coverage.unit |
234 | -else |
235 | - python3 -m unittest |
236 | -fi |
237 | - |
238 | -if [ -z "$SNAPCRAFT_TESTS_SKIP_PLAINBOX" ]; then |
239 | -( |
240 | +parseargs(){ |
241 | + if [[ "$#" -eq 0 ]] || [[ "$1" == "all" ]]; then |
242 | + export RUN_UNIT="true" |
243 | + export RUN_PLAINBOX="true" |
244 | + export PLAINBOX_TEST_PLANS="normal" |
245 | + else |
246 | + if [ "$1" == "unit" ] ; then |
247 | + export RUN_UNIT="true" |
248 | + elif [ "$1" == "plainbox" ] ; then |
249 | + export RUN_PLAINBOX="true" |
250 | + if [ "$#" -gt 1 ]; then |
251 | + export PLAINBOX_TEST_PLANS="$2" |
252 | + else |
253 | + export PLAINBOX_TEST_PLANS="normal" |
254 | + fi |
255 | + else |
256 | + echo "Not recognized option, should be one of all, unit or plainbox" |
257 | + exit 1 |
258 | + fi |
259 | + fi |
260 | +} |
261 | + |
262 | +run_unit_tests(){ |
263 | + SRC_PATHS="bin snapcraft snapcraft/tests" |
264 | + |
265 | + # These three checks could easily be done with flake8 in one shot if |
266 | + # we had python3-flake8 provide flake8 |
267 | + # Ignore 501 (line-too-long) |
268 | + pep8 $SRC_PATHS --ignore=E501 |
269 | + |
270 | + pyflakes3 $SRC_PATHS |
271 | + |
272 | + # mccabe in 'warning' mode as we have high complexity |
273 | + mccabe_list= |
274 | + for unit in $(find . -type f -name '*.py') |
275 | + do |
276 | + output=$(python3 -m mccabe --min 10 "$unit") |
277 | + [ -n "$output" ] && mccabe_list="- $unit:\n $output\n$mccabe_list" |
278 | + done |
279 | + |
280 | + if [ -n "$mccabe_list" ]; then |
281 | + echo -e "\e[1;31mThe project has gotten complex\e[0m." |
282 | + echo "Here's the list of units exceeding 10:" |
283 | + echo -e "$mccabe_list" |
284 | + fi |
285 | + |
286 | + if which python3-coverage >/dev/null 2>&1; then |
287 | + python3-coverage erase |
288 | + python3-coverage run --branch --source snapcraft -m unittest |
289 | + mv .coverage .coverage.unit |
290 | + else |
291 | + python3 -m unittest |
292 | + fi |
293 | +} |
294 | + |
295 | +run_plainbox(){ |
296 | # well, well, what can we do |
297 | if ! which plainbox >/dev/null; then |
298 | cat <<EOF |
299 | @@ -73,8 +96,17 @@ |
300 | |
301 | # Go to the plainbox provider of snapcraft tests |
302 | cd integration-tests |
303 | - ./runtests.sh |
304 | -) |
305 | + ./runtests.sh $PLAINBOX_TEST_PLANS |
306 | +} |
307 | + |
308 | +parseargs "$@" |
309 | + |
310 | +if [ ! -z "$RUN_UNIT" ]; then |
311 | + run_unit_tests |
312 | +fi |
313 | + |
314 | +if [ -z "$SNAPCRAFT_TESTS_SKIP_PLAINBOX" ] && [ ! -z "$RUN_PLAINBOX" ] ; then |
315 | + run_plainbox |
316 | fi |
317 | |
318 | if which python3-coverage >/dev/null 2>&1; then |
Thanks, this is a good start, looping over everything in examples will do the initial trick. In the future I will split this out a bit and use something a bit more custom for each checking if stage and snap have the right set of files for example.
The reason I set to needs fixing though is because of 'run_examples_ tests.sh' , it seems there is too much duplicate code here and in runtests.sh, it also introduces one more driver for them.
I'd like to see this at least driven by an envvar or command arg
./runtests.sh all (default)
./runtests.sh unit
./runtest.sh plainbox
./runtest.sh plainbox examples|normal
Thoughts?