Status: | Merged |
---|---|
Merged at revision: | 490 |
Proposed branch: | lp:~cjwatson/storm/tox |
Merge into: | lp:storm |
Diff against target: |
483 lines (+51/-323) 8 files modified
.bzrignore (+1/-0) MANIFEST.in (+1/-1) Makefile (+4/-13) NEWS (+1/-0) dev/ubuntu-deps (+4/-1) ez_setup.py (+0/-284) setup.py (+25/-24) tox.ini (+15/-0) |
To merge this branch: | bzr merge lp:~cjwatson/storm/tox |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Simon Poirier (community) | Approve | ||
Review via email: mp+368142@code.launchpad.net |
Commit message
Add tox testing support.
Description of the change
My goal here is to make it easier to test with various versions of dependencies, and in particular to make it easier to test with multiple Python versions in order to facilitate a port to Python 3.
This has IMO already pulled its weight by uncovering https:/
To post a comment you must log in.
Revision history for this message
Colin Watson (cjwatson) wrote : | # |
What failures do you see? I have a couple of associated MPs that fix some failures, but I'd need to know details of your problems to tell whether they're relevant.
Revision history for this message
Simon Poirier (simpoir) wrote : | # |
checked with the other branches +1
review:
Approve
Preview Diff
[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1 | === modified file '.bzrignore' |
2 | --- .bzrignore 2011-10-18 11:20:25 +0000 |
3 | +++ .bzrignore 2019-05-30 13:25:31 +0000 |
4 | @@ -10,6 +10,7 @@ |
5 | debian/python-storm.prerm.debhelper |
6 | debian/python-storm.substvars |
7 | apidoc |
8 | +.tox |
9 | _trial_temp |
10 | *.egg |
11 | TAGS |
12 | |
13 | === modified file 'MANIFEST.in' |
14 | --- MANIFEST.in 2011-10-25 10:47:06 +0000 |
15 | +++ MANIFEST.in 2019-05-30 13:25:31 +0000 |
16 | @@ -1,4 +1,4 @@ |
17 | recursive-include storm *.py *.c *.zcml |
18 | recursive-include tests *.py *.txt |
19 | |
20 | -include MANIFEST.in LICENSE README TODO NEWS Makefile setup.cfg test ez_setup.py |
21 | +include MANIFEST.in LICENSE README TODO NEWS Makefile setup.cfg test tox.ini |
22 | |
23 | === modified file 'Makefile' |
24 | --- Makefile 2016-03-10 12:04:01 +0000 |
25 | +++ Makefile 2019-05-30 13:25:31 +0000 |
26 | @@ -1,15 +1,8 @@ |
27 | PYTHON ?= python |
28 | PYDOCTOR ?= pydoctor |
29 | -PGPORT ?= 5432 |
30 | |
31 | TEST_COMMAND = $(PYTHON) setup.py test |
32 | |
33 | -STORM_POSTGRES_URI = postgres:storm_test |
34 | -STORM_POSTGRES_HOST_URI = postgres://localhost:$(PGPORT)/storm_test |
35 | - |
36 | -export STORM_POSTGRES_URI |
37 | -export STORM_POSTGRES_HOST_URI |
38 | - |
39 | all: build |
40 | |
41 | build: |
42 | @@ -19,13 +12,10 @@ |
43 | $(TEST_COMMAND) --quiet --dry-run |
44 | |
45 | check: |
46 | - @ # Run the tests once with C extensions and once without them. |
47 | - STORM_CEXTENSIONS=0 $(TEST_COMMAND) |
48 | - STORM_CEXTENSIONS=1 $(TEST_COMMAND) |
49 | + tox |
50 | |
51 | -check-with-trial: develop |
52 | - STORM_TEST_RUNNER=trial STORM_CEXTENSIONS=0 $(PYTHON) test |
53 | - STORM_TEST_RUNNER=trial STORM_CEXTENSIONS=1 $(PYTHON) test |
54 | +check-with-trial: |
55 | + STORM_TEST_RUNNER=trial tox |
56 | |
57 | doc: |
58 | $(PYDOCTOR) --make-html --html-output apidoc --add-package storm |
59 | @@ -41,6 +31,7 @@ |
60 | rm -rf debian/files |
61 | rm -rf debian/python-storm |
62 | rm -rf debian/python-storm.* |
63 | + rm -rf .tox |
64 | rm -rf *.egg |
65 | rm -rf _trial_temp |
66 | find . -name "*.so" -type f -exec rm -f {} \; |
67 | |
68 | === modified file 'NEWS' |
69 | --- NEWS 2016-03-01 13:09:28 +0000 |
70 | +++ NEWS 2019-05-30 13:25:31 +0000 |
71 | @@ -7,6 +7,7 @@ |
72 | - A new storm.schema.sharding.Sharding class has been added to the schema |
73 | package, which can apply patches "in parallel" against a set of stores. See |
74 | the module docstring for more information. |
75 | +- Added tox testing support. |
76 | |
77 | Bug fixes |
78 | --------- |
79 | |
80 | === modified file 'dev/ubuntu-deps' |
81 | --- dev/ubuntu-deps 2016-03-10 12:04:01 +0000 |
82 | +++ dev/ubuntu-deps 2019-05-30 13:25:31 +0000 |
83 | @@ -8,7 +8,9 @@ |
84 | apt_get install --no-install-recommends \ |
85 | postgresql \ |
86 | pgbouncer \ |
87 | + libpq-dev \ |
88 | build-essential \ |
89 | + python-dev \ |
90 | python-fixtures \ |
91 | python-psycopg2 \ |
92 | python-testresources \ |
93 | @@ -16,7 +18,8 @@ |
94 | python-twisted \ |
95 | python-zope.component \ |
96 | python-zope.security \ |
97 | - python-setuptools |
98 | + python-setuptools \ |
99 | + tox |
100 | |
101 | |
102 | echo " * All Done!" |
103 | |
104 | === removed file 'ez_setup.py' |
105 | --- ez_setup.py 2011-10-18 13:59:56 +0000 |
106 | +++ ez_setup.py 1970-01-01 00:00:00 +0000 |
107 | @@ -1,284 +0,0 @@ |
108 | -#!python |
109 | -"""Bootstrap setuptools installation |
110 | - |
111 | -If you want to use setuptools in your package's setup.py, just include this |
112 | -file in the same directory with it, and add this to the top of your setup.py:: |
113 | - |
114 | - from ez_setup import use_setuptools |
115 | - use_setuptools() |
116 | - |
117 | -If you want to require a specific version of setuptools, set a download |
118 | -mirror, or use an alternate download directory, you can do so by supplying |
119 | -the appropriate options to ``use_setuptools()``. |
120 | - |
121 | -This file can also be run as a script to install or upgrade setuptools. |
122 | -""" |
123 | -import sys |
124 | -DEFAULT_VERSION = "0.6c11" |
125 | -DEFAULT_URL = "http://pypi.python.org/packages/%s/s/setuptools/" % sys.version[:3] |
126 | - |
127 | -md5_data = { |
128 | - 'setuptools-0.6b1-py2.3.egg': '8822caf901250d848b996b7f25c6e6ca', |
129 | - 'setuptools-0.6b1-py2.4.egg': 'b79a8a403e4502fbb85ee3f1941735cb', |
130 | - 'setuptools-0.6b2-py2.3.egg': '5657759d8a6d8fc44070a9d07272d99b', |
131 | - 'setuptools-0.6b2-py2.4.egg': '4996a8d169d2be661fa32a6e52e4f82a', |
132 | - 'setuptools-0.6b3-py2.3.egg': 'bb31c0fc7399a63579975cad9f5a0618', |
133 | - 'setuptools-0.6b3-py2.4.egg': '38a8c6b3d6ecd22247f179f7da669fac', |
134 | - 'setuptools-0.6b4-py2.3.egg': '62045a24ed4e1ebc77fe039aa4e6f7e5', |
135 | - 'setuptools-0.6b4-py2.4.egg': '4cb2a185d228dacffb2d17f103b3b1c4', |
136 | - 'setuptools-0.6c1-py2.3.egg': 'b3f2b5539d65cb7f74ad79127f1a908c', |
137 | - 'setuptools-0.6c1-py2.4.egg': 'b45adeda0667d2d2ffe14009364f2a4b', |
138 | - 'setuptools-0.6c10-py2.3.egg': 'ce1e2ab5d3a0256456d9fc13800a7090', |
139 | - 'setuptools-0.6c10-py2.4.egg': '57d6d9d6e9b80772c59a53a8433a5dd4', |
140 | - 'setuptools-0.6c10-py2.5.egg': 'de46ac8b1c97c895572e5e8596aeb8c7', |
141 | - 'setuptools-0.6c10-py2.6.egg': '58ea40aef06da02ce641495523a0b7f5', |
142 | - 'setuptools-0.6c11-py2.3.egg': '2baeac6e13d414a9d28e7ba5b5a596de', |
143 | - 'setuptools-0.6c11-py2.4.egg': 'bd639f9b0eac4c42497034dec2ec0c2b', |
144 | - 'setuptools-0.6c11-py2.5.egg': '64c94f3bf7a72a13ec83e0b24f2749b2', |
145 | - 'setuptools-0.6c11-py2.6.egg': 'bfa92100bd772d5a213eedd356d64086', |
146 | - 'setuptools-0.6c2-py2.3.egg': 'f0064bf6aa2b7d0f3ba0b43f20817c27', |
147 | - 'setuptools-0.6c2-py2.4.egg': '616192eec35f47e8ea16cd6a122b7277', |
148 | - 'setuptools-0.6c3-py2.3.egg': 'f181fa125dfe85a259c9cd6f1d7b78fa', |
149 | - 'setuptools-0.6c3-py2.4.egg': 'e0ed74682c998bfb73bf803a50e7b71e', |
150 | - 'setuptools-0.6c3-py2.5.egg': 'abef16fdd61955514841c7c6bd98965e', |
151 | - 'setuptools-0.6c4-py2.3.egg': 'b0b9131acab32022bfac7f44c5d7971f', |
152 | - 'setuptools-0.6c4-py2.4.egg': '2a1f9656d4fbf3c97bf946c0a124e6e2', |
153 | - 'setuptools-0.6c4-py2.5.egg': '8f5a052e32cdb9c72bcf4b5526f28afc', |
154 | - 'setuptools-0.6c5-py2.3.egg': 'ee9fd80965da04f2f3e6b3576e9d8167', |
155 | - 'setuptools-0.6c5-py2.4.egg': 'afe2adf1c01701ee841761f5bcd8aa64', |
156 | - 'setuptools-0.6c5-py2.5.egg': 'a8d3f61494ccaa8714dfed37bccd3d5d', |
157 | - 'setuptools-0.6c6-py2.3.egg': '35686b78116a668847237b69d549ec20', |
158 | - 'setuptools-0.6c6-py2.4.egg': '3c56af57be3225019260a644430065ab', |
159 | - 'setuptools-0.6c6-py2.5.egg': 'b2f8a7520709a5b34f80946de5f02f53', |
160 | - 'setuptools-0.6c7-py2.3.egg': '209fdf9adc3a615e5115b725658e13e2', |
161 | - 'setuptools-0.6c7-py2.4.egg': '5a8f954807d46a0fb67cf1f26c55a82e', |
162 | - 'setuptools-0.6c7-py2.5.egg': '45d2ad28f9750e7434111fde831e8372', |
163 | - 'setuptools-0.6c8-py2.3.egg': '50759d29b349db8cfd807ba8303f1902', |
164 | - 'setuptools-0.6c8-py2.4.egg': 'cba38d74f7d483c06e9daa6070cce6de', |
165 | - 'setuptools-0.6c8-py2.5.egg': '1721747ee329dc150590a58b3e1ac95b', |
166 | - 'setuptools-0.6c9-py2.3.egg': 'a83c4020414807b496e4cfbe08507c03', |
167 | - 'setuptools-0.6c9-py2.4.egg': '260a2be2e5388d66bdaee06abec6342a', |
168 | - 'setuptools-0.6c9-py2.5.egg': 'fe67c3e5a17b12c0e7c541b7ea43a8e6', |
169 | - 'setuptools-0.6c9-py2.6.egg': 'ca37b1ff16fa2ede6e19383e7b59245a', |
170 | -} |
171 | - |
172 | -import sys, os |
173 | -try: from hashlib import md5 |
174 | -except ImportError: from md5 import md5 |
175 | - |
176 | -def _validate_md5(egg_name, data): |
177 | - if egg_name in md5_data: |
178 | - digest = md5(data).hexdigest() |
179 | - if digest != md5_data[egg_name]: |
180 | - print >>sys.stderr, ( |
181 | - "md5 validation of %s failed! (Possible download problem?)" |
182 | - % egg_name |
183 | - ) |
184 | - sys.exit(2) |
185 | - return data |
186 | - |
187 | -def use_setuptools( |
188 | - version=DEFAULT_VERSION, download_base=DEFAULT_URL, to_dir=os.curdir, |
189 | - download_delay=15 |
190 | -): |
191 | - """Automatically find/download setuptools and make it available on sys.path |
192 | - |
193 | - `version` should be a valid setuptools version number that is available |
194 | - as an egg for download under the `download_base` URL (which should end with |
195 | - a '/'). `to_dir` is the directory where setuptools will be downloaded, if |
196 | - it is not already available. If `download_delay` is specified, it should |
197 | - be the number of seconds that will be paused before initiating a download, |
198 | - should one be required. If an older version of setuptools is installed, |
199 | - this routine will print a message to ``sys.stderr`` and raise SystemExit in |
200 | - an attempt to abort the calling script. |
201 | - """ |
202 | - was_imported = 'pkg_resources' in sys.modules or 'setuptools' in sys.modules |
203 | - def do_download(): |
204 | - egg = download_setuptools(version, download_base, to_dir, download_delay) |
205 | - sys.path.insert(0, egg) |
206 | - import setuptools; setuptools.bootstrap_install_from = egg |
207 | - try: |
208 | - import pkg_resources |
209 | - except ImportError: |
210 | - return do_download() |
211 | - try: |
212 | - pkg_resources.require("setuptools>="+version); return |
213 | - except pkg_resources.VersionConflict, e: |
214 | - if was_imported: |
215 | - print >>sys.stderr, ( |
216 | - "The required version of setuptools (>=%s) is not available, and\n" |
217 | - "can't be installed while this script is running. Please install\n" |
218 | - " a more recent version first, using 'easy_install -U setuptools'." |
219 | - "\n\n(Currently using %r)" |
220 | - ) % (version, e.args[0]) |
221 | - sys.exit(2) |
222 | - except pkg_resources.DistributionNotFound: |
223 | - pass |
224 | - |
225 | - del pkg_resources, sys.modules['pkg_resources'] # reload ok |
226 | - return do_download() |
227 | - |
228 | -def download_setuptools( |
229 | - version=DEFAULT_VERSION, download_base=DEFAULT_URL, to_dir=os.curdir, |
230 | - delay = 15 |
231 | -): |
232 | - """Download setuptools from a specified location and return its filename |
233 | - |
234 | - `version` should be a valid setuptools version number that is available |
235 | - as an egg for download under the `download_base` URL (which should end |
236 | - with a '/'). `to_dir` is the directory where the egg will be downloaded. |
237 | - `delay` is the number of seconds to pause before an actual download attempt. |
238 | - """ |
239 | - import urllib2, shutil |
240 | - egg_name = "setuptools-%s-py%s.egg" % (version,sys.version[:3]) |
241 | - url = download_base + egg_name |
242 | - saveto = os.path.join(to_dir, egg_name) |
243 | - src = dst = None |
244 | - if not os.path.exists(saveto): # Avoid repeated downloads |
245 | - try: |
246 | - from distutils import log |
247 | - if delay: |
248 | - log.warn(""" |
249 | ---------------------------------------------------------------------------- |
250 | -This script requires setuptools version %s to run (even to display |
251 | -help). I will attempt to download it for you (from |
252 | -%s), but |
253 | -you may need to enable firewall access for this script first. |
254 | -I will start the download in %d seconds. |
255 | - |
256 | -(Note: if this machine does not have network access, please obtain the file |
257 | - |
258 | - %s |
259 | - |
260 | -and place it in this directory before rerunning this script.) |
261 | ----------------------------------------------------------------------------""", |
262 | - version, download_base, delay, url |
263 | - ); from time import sleep; sleep(delay) |
264 | - log.warn("Downloading %s", url) |
265 | - src = urllib2.urlopen(url) |
266 | - # Read/write all in one block, so we don't create a corrupt file |
267 | - # if the download is interrupted. |
268 | - data = _validate_md5(egg_name, src.read()) |
269 | - dst = open(saveto,"wb"); dst.write(data) |
270 | - finally: |
271 | - if src: src.close() |
272 | - if dst: dst.close() |
273 | - return os.path.realpath(saveto) |
274 | - |
275 | - |
276 | - |
277 | - |
278 | - |
279 | - |
280 | - |
281 | - |
282 | - |
283 | - |
284 | - |
285 | - |
286 | - |
287 | - |
288 | - |
289 | - |
290 | - |
291 | - |
292 | - |
293 | - |
294 | - |
295 | - |
296 | - |
297 | - |
298 | - |
299 | - |
300 | - |
301 | - |
302 | - |
303 | - |
304 | - |
305 | - |
306 | - |
307 | - |
308 | - |
309 | - |
310 | -def main(argv, version=DEFAULT_VERSION): |
311 | - """Install or upgrade setuptools and EasyInstall""" |
312 | - try: |
313 | - import setuptools |
314 | - except ImportError: |
315 | - egg = None |
316 | - try: |
317 | - egg = download_setuptools(version, delay=0) |
318 | - sys.path.insert(0,egg) |
319 | - from setuptools.command.easy_install import main |
320 | - return main(list(argv)+[egg]) # we're done here |
321 | - finally: |
322 | - if egg and os.path.exists(egg): |
323 | - os.unlink(egg) |
324 | - else: |
325 | - if setuptools.__version__ == '0.0.1': |
326 | - print >>sys.stderr, ( |
327 | - "You have an obsolete version of setuptools installed. Please\n" |
328 | - "remove it from your system entirely before rerunning this script." |
329 | - ) |
330 | - sys.exit(2) |
331 | - |
332 | - req = "setuptools>="+version |
333 | - import pkg_resources |
334 | - try: |
335 | - pkg_resources.require(req) |
336 | - except pkg_resources.VersionConflict: |
337 | - try: |
338 | - from setuptools.command.easy_install import main |
339 | - except ImportError: |
340 | - from easy_install import main |
341 | - main(list(argv)+[download_setuptools(delay=0)]) |
342 | - sys.exit(0) # try to force an exit |
343 | - else: |
344 | - if argv: |
345 | - from setuptools.command.easy_install import main |
346 | - main(argv) |
347 | - else: |
348 | - print "Setuptools version",version,"or greater has been installed." |
349 | - print '(Run "ez_setup.py -U setuptools" to reinstall or upgrade.)' |
350 | - |
351 | -def update_md5(filenames): |
352 | - """Update our built-in md5 registry""" |
353 | - |
354 | - import re |
355 | - |
356 | - for name in filenames: |
357 | - base = os.path.basename(name) |
358 | - f = open(name,'rb') |
359 | - md5_data[base] = md5(f.read()).hexdigest() |
360 | - f.close() |
361 | - |
362 | - data = [" %r: %r,\n" % it for it in md5_data.items()] |
363 | - data.sort() |
364 | - repl = "".join(data) |
365 | - |
366 | - import inspect |
367 | - srcfile = inspect.getsourcefile(sys.modules[__name__]) |
368 | - f = open(srcfile, 'rb'); src = f.read(); f.close() |
369 | - |
370 | - match = re.search("\nmd5_data = {\n([^}]+)}", src) |
371 | - if not match: |
372 | - print >>sys.stderr, "Internal error!" |
373 | - sys.exit(2) |
374 | - |
375 | - src = src[:match.start(1)] + repl + src[match.end(1):] |
376 | - f = open(srcfile,'w') |
377 | - f.write(src) |
378 | - f.close() |
379 | - |
380 | - |
381 | -if __name__=='__main__': |
382 | - if len(sys.argv)>2 and sys.argv[1]=='--md5update': |
383 | - update_md5(sys.argv[2:]) |
384 | - else: |
385 | - main(sys.argv[1:]) |
386 | - |
387 | - |
388 | - |
389 | - |
390 | - |
391 | - |
392 | |
393 | === modified file 'setup.py' |
394 | --- setup.py 2016-03-01 13:09:28 +0000 |
395 | +++ setup.py 2019-05-30 13:25:31 +0000 |
396 | @@ -2,9 +2,6 @@ |
397 | import os |
398 | import re |
399 | |
400 | -import ez_setup |
401 | -ez_setup.use_setuptools() |
402 | - |
403 | from setuptools import setup, Extension, find_packages |
404 | |
405 | |
406 | @@ -19,6 +16,28 @@ |
407 | open("storm/__init__.py").read()).group(1) |
408 | |
409 | |
410 | +tests_require = [ |
411 | + # Versions based on Lucid, where packaged. |
412 | + "fixtures >= 0.3.5", |
413 | + # pgbouncer (the Python module) is not yet packaged in Ubuntu. |
414 | + "pgbouncer >= 0.0.7", |
415 | + "psycopg2 >= 2.0.13", |
416 | + "testresources >= 0.2.4", |
417 | + "testtools >= 0.9.8", |
418 | + # timeline is not yet packaged in Ubuntu. |
419 | + "timeline >= 0.0.2", |
420 | + "transaction >= 1.0.0", |
421 | + "twisted >= 10.0.0", |
422 | + "zope.component >= 3.8.0", |
423 | + # zope.component 3.11.0 requires a version of zope.interface that no |
424 | + # version of Ubuntu yet packages. The following rule exists for the |
425 | + # sake of convenience rather than necessity, for the situation where |
426 | + # zope.interface is installed via a package but zope.component is not. |
427 | + "zope.component < 3.11.0", |
428 | + "zope.security >= 3.7.2", |
429 | + ] |
430 | + |
431 | + |
432 | setup( |
433 | name="storm", |
434 | version=VERSION, |
435 | @@ -50,25 +69,7 @@ |
436 | # warning) by distutils. |
437 | include_package_data=True, |
438 | zip_safe=False, |
439 | - test_suite = "tests.find_tests", |
440 | - tests_require=[ |
441 | - # Versions based on Lucid, where packaged. |
442 | - "fixtures >= 0.3.5", |
443 | - # pgbouncer (the Python module) is not yet packaged in Ubuntu. |
444 | - "pgbouncer >= 0.0.7", |
445 | - "psycopg2 >= 2.0.13", |
446 | - "testresources >= 0.2.4", |
447 | - "testtools >= 0.9.8", |
448 | - # timeline is not yet packaged in Ubuntu. |
449 | - "timeline >= 0.0.2", |
450 | - "transaction >= 1.0.0", |
451 | - "twisted >= 10.0.0", |
452 | - "zope.component >= 3.8.0", |
453 | - # zope.component 3.11.0 requires a version of zope.interface that no |
454 | - # version of Ubuntu yet packages. The following rule exists for the |
455 | - # sake of convenience rather than necessity, for the situation where |
456 | - # zope.interface is installed via a package but zope.component is not. |
457 | - "zope.component < 3.11.0", |
458 | - "zope.security >= 3.7.2", |
459 | - ], |
460 | + test_suite="tests.find_tests", |
461 | + tests_require=tests_require, |
462 | + extras_require={"test": tests_require}, |
463 | ) |
464 | |
465 | === added file 'tox.ini' |
466 | --- tox.ini 1970-01-01 00:00:00 +0000 |
467 | +++ tox.ini 2019-05-30 13:25:31 +0000 |
468 | @@ -0,0 +1,15 @@ |
469 | +[tox] |
470 | +envlist = |
471 | + py27-{cextensions,nocextensions} |
472 | + |
473 | +[testenv] |
474 | +deps = |
475 | + .[test] |
476 | +passenv = STORM_TEST_RUNNER USER |
477 | +setenv = |
478 | + STORM_POSTGRES_URI = postgres:storm_test |
479 | + STORM_POSTGRES_HOST_URI = postgres://localhost:5432/storm_test |
480 | + cextensions: STORM_CEXTENSIONS = 0 |
481 | + nocextensions: STORM_CEXTENSIONS = 1 |
482 | +commands = |
483 | + python test {posargs} |
I"m generally ok with this change, but I do get some failures when running test under tox+py27 vs py27 on a fresh container. Am I missing something or will this be in a follow-up branch?