Merge lp:~gary/zc.buildout/python-support-6-egg-control into lp:zc.buildout
- python-support-6-egg-control
- Merge into trunk
Status: | Needs review |
---|---|
Proposed branch: | lp:~gary/zc.buildout/python-support-6-egg-control |
Merge into: | lp:zc.buildout |
Prerequisite: | lp:~gary/zc.buildout/python-support-5-initial-egg-control |
Diff against target: |
683 lines (+426/-28) 12 files modified
CHANGES.txt (+6/-5) dev.py (+3/-1) src/zc/buildout/buildout.py (+7/-4) src/zc/buildout/buildout.txt (+7/-2) src/zc/buildout/easy_install.py (+1/-1) src/zc/buildout/easy_install.txt (+1/-1) src/zc/buildout/testing.py (+10/-7) src/zc/buildout/tests.py (+91/-2) z3c.recipe.scripts_/src/z3c/recipe/scripts/README.txt (+126/-3) z3c.recipe.scripts_/src/z3c/recipe/scripts/scripts.py (+5/-0) z3c.recipe.scripts_/src/z3c/recipe/scripts/tests.py (+163/-2) zc.recipe.egg_/src/zc/recipe/egg/egg.py (+6/-0) |
To merge this branch: | bzr merge lp:~gary/zc.buildout/python-support-6-egg-control |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Francis J. Lacoste (community) | Approve | ||
Review via email: mp+20013@code.launchpad.net |
Commit message
Description of the change
- 557. By Gary Poster
-
make the buildout script itself safe for a Python with site packages.
Gary Poster (gary) wrote : | # |
I made a small but important change in r557. The buildout script itself is now created with sitepackage_
I have not come up with an automated test for why this is necessary. We do have plenty of automated tests showing that it works as well as the ``scripts`` function for this purpose. I will spend a bit more time determining if I can come up with a test that shows why this change is necessary, but I'm hopeful it is acceptable as is.
I also determined that the do_build change I mentioned in the initial review request was unnecessary. I removed it.
- 558. By Gary Poster
-
add test for recent fix for buildout
Gary Poster (gary) wrote : | # |
I was able to figure out a not-too-horrible way of testing the necessity of the change in r557. I added it to r558.
Francis J. Lacoste (flacoste) wrote : | # |
Nothing to say on this branch, apart thanks for the typos fixes!
- 559. By Gary Poster
-
merge from gary-5
Unmerged revisions
- 559. By Gary Poster
-
merge from gary-5
- 558. By Gary Poster
-
add test for recent fix for buildout
- 557. By Gary Poster
-
make the buildout script itself safe for a Python with site packages.
- 556. By Gary Poster
-
merge from gary-5 <- gary-4
- 555. By Gary Poster
-
merge from gary-5 <- gary-4
- 554. By Gary Poster
-
make tests pass in Windows.
- 553. By Gary Poster
-
update docs for changes in this branch.
- 552. By Gary Poster
-
merge gary-5 changes
- 551. By Gary Poster
-
integrate ability to control what eggs are accepted from site-packages into z3c.recipe.scripts. Without this, the ability to use z3c.recipe.scripts without site-packages was broken.
- 550. By Gary Poster
-
support limiting packages from site-packages
Preview Diff
1 | === modified file 'CHANGES.txt' |
2 | --- CHANGES.txt 2010-02-24 23:23:14 +0000 |
3 | +++ CHANGES.txt 2010-02-24 23:23:14 +0000 |
4 | @@ -15,11 +15,12 @@ |
5 | without code installed in site-packages. It keeps its previous behavior in |
6 | order to provide backwards compatibility. |
7 | |
8 | - (Note that this branch is incomplete in its implementation of this feature: |
9 | - if eggs are in installed in site-packages but you do not want to use |
10 | - site-packages, the eggs will drag in site-packages even if you try to |
11 | - exclude it. This is addressed in subsequent branches in the series of |
12 | - which this one is a part.) |
13 | + The z3c.recipe.scripts recipe allows you to control how you use the |
14 | + code in site-packages. You can exclude it entirely; allow eggs in it |
15 | + to fulfill package dependencies declared in setup.py and buildout |
16 | + configuration; allow it to be available but not used to fulfill |
17 | + dependencies declared in setup.py or buildout configuration; or only |
18 | + allow certain eggs in site-packages to fulfill dependencies. |
19 | |
20 | - Added new function, ``zc.buildout.easy_install.sitepackage_safe_scripts``, |
21 | to generate scripts and interpreter. It produces a full-featured |
22 | |
23 | === modified file 'dev.py' |
24 | --- dev.py 2010-02-24 23:23:14 +0000 |
25 | +++ dev.py 2010-02-24 23:23:14 +0000 |
26 | @@ -40,10 +40,12 @@ |
27 | |
28 | import pkg_resources |
29 | |
30 | +env = os.environ.copy() # Windows needs yet-to-be-determined values from this. |
31 | +env['PYTHONPATH'] = os.path.dirname(pkg_resources.__file__) |
32 | subprocess.Popen( |
33 | [sys.executable] + |
34 | ['setup.py', '-q', 'develop', '-m', '-x', '-d', 'develop-eggs'], |
35 | - env = {'PYTHONPATH': os.path.dirname(pkg_resources.__file__)}).wait() |
36 | + env=env).wait() |
37 | |
38 | pkg_resources.working_set.add_entry('src') |
39 | |
40 | |
41 | === modified file 'src/zc/buildout/buildout.py' |
42 | --- src/zc/buildout/buildout.py 2010-02-24 23:23:14 +0000 |
43 | +++ src/zc/buildout/buildout.py 2010-02-24 23:23:14 +0000 |
44 | @@ -391,9 +391,12 @@ |
45 | # Create buildout script |
46 | ws = pkg_resources.WorkingSet(entries) |
47 | ws.require('zc.buildout') |
48 | - zc.buildout.easy_install.scripts( |
49 | - ['zc.buildout'], ws, options['executable'], |
50 | - options['bin-directory']) |
51 | + partsdir = os.path.join(options['parts-directory'], 'buildout') |
52 | + if not os.path.exists(partsdir): |
53 | + os.mkdir(partsdir) |
54 | + zc.buildout.easy_install.sitepackage_safe_scripts( |
55 | + options['bin-directory'], ws, options['executable'], partsdir, |
56 | + reqs=['zc.buildout']) |
57 | |
58 | init = bootstrap |
59 | |
60 | @@ -563,7 +566,7 @@ |
61 | if installed_files is None: |
62 | self._logger.warning( |
63 | "The %s install returned None. A path or " |
64 | - "iterable os paths should be returned.", |
65 | + "iterable of paths should be returned.", |
66 | part) |
67 | installed_files = () |
68 | elif isinstance(installed_files, str): |
69 | |
70 | === modified file 'src/zc/buildout/buildout.txt' |
71 | --- src/zc/buildout/buildout.txt 2010-02-24 23:23:14 +0000 |
72 | +++ src/zc/buildout/buildout.txt 2010-02-24 23:23:14 +0000 |
73 | @@ -56,10 +56,9 @@ |
74 | - setuptools-0.6-py2.4.egg |
75 | - zc.buildout-1.0-py2.4.egg |
76 | |
77 | -The develop-eggs and parts directories are initially empty: |
78 | +The develop-eggs directory is initially empty: |
79 | |
80 | >>> ls(sample_buildout, 'develop-eggs') |
81 | - >>> ls(sample_buildout, 'parts') |
82 | |
83 | The develop-eggs directory holds egg links for software being |
84 | developed in the buildout. We separate develop-eggs and other eggs to |
85 | @@ -69,6 +68,12 @@ |
86 | allows larger buildouts to be set up much more quickly and saves disk |
87 | space. |
88 | |
89 | +The parts directory just contains some helpers for the buildout script |
90 | +itself. |
91 | + |
92 | + >>> ls(sample_buildout, 'parts') |
93 | + d buildout |
94 | + |
95 | The parts directory provides an area where recipes can install |
96 | part data. For example, if we built a custom Python, we would |
97 | install it in the part directory. Part data is stored in a |
98 | |
99 | === modified file 'src/zc/buildout/easy_install.py' |
100 | --- src/zc/buildout/easy_install.py 2010-02-24 23:23:14 +0000 |
101 | +++ src/zc/buildout/easy_install.py 2010-02-24 23:23:14 +0000 |
102 | @@ -197,7 +197,7 @@ |
103 | |
104 | if is_win32: |
105 | # work around spawn lamosity on windows |
106 | - # XXX need safe quoting (see the subproces.list2cmdline) and test |
107 | + # XXX need safe quoting (see the subprocess.list2cmdline) and test |
108 | def _safe_arg(arg): |
109 | return '"%s"' % arg |
110 | else: |
111 | |
112 | === modified file 'src/zc/buildout/easy_install.txt' |
113 | --- src/zc/buildout/easy_install.txt 2010-02-24 23:23:14 +0000 |
114 | +++ src/zc/buildout/easy_install.txt 2010-02-24 23:23:14 +0000 |
115 | @@ -998,7 +998,7 @@ |
116 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ |
117 | |
118 | The newer function for creating scripts is ``sitepackage_safe_scripts``. |
119 | - It has the same basic functionality as the ``scripts`` function: it can |
120 | +It has the same basic functionality as the ``scripts`` function: it can |
121 | create scripts to run arbitrary entry points, and to run a Python |
122 | interpreter. The following are the differences from a user's |
123 | perspective. |
124 | |
125 | === modified file 'src/zc/buildout/testing.py' |
126 | --- src/zc/buildout/testing.py 2010-02-24 23:23:14 +0000 |
127 | +++ src/zc/buildout/testing.py 2010-02-24 23:23:14 +0000 |
128 | @@ -244,7 +244,7 @@ |
129 | for name, value in values.items(): |
130 | getattr(zc.buildout.easy_install, name)(value) |
131 | |
132 | -def make_buildout(): |
133 | +def make_buildout(executable=None): |
134 | """Make a buildout that uses this version of zc.buildout.""" |
135 | # Create a basic buildout.cfg to avoid a warning from buildout. |
136 | open('buildout.cfg', 'w').write( |
137 | @@ -254,13 +254,16 @@ |
138 | # a Buildout will force the Buildout's defaults on the installer). |
139 | installer_values = get_installer_values() |
140 | # Use the buildout bootstrap command to create a buildout |
141 | + config = [ |
142 | + ('buildout', 'log-level', 'WARNING'), |
143 | + # trick bootstrap into putting the buildout develop egg |
144 | + # in the eggs dir. |
145 | + ('buildout', 'develop-eggs-directory', 'eggs'), |
146 | + ] |
147 | + if executable is not None: |
148 | + config.append(('buildout', 'executable', executable)) |
149 | zc.buildout.buildout.Buildout( |
150 | - 'buildout.cfg', |
151 | - [('buildout', 'log-level', 'WARNING'), |
152 | - # trick bootstrap into putting the buildout develop egg |
153 | - # in the eggs dir. |
154 | - ('buildout', 'develop-eggs-directory', 'eggs'), |
155 | - ], |
156 | + 'buildout.cfg', config, |
157 | user_defaults=False, |
158 | ).bootstrap([]) |
159 | # Create the develop-eggs dir, which didn't get created the usual |
160 | |
161 | === modified file 'src/zc/buildout/tests.py' |
162 | --- src/zc/buildout/tests.py 2010-02-24 23:23:14 +0000 |
163 | +++ src/zc/buildout/tests.py 2010-02-24 23:23:14 +0000 |
164 | @@ -420,7 +420,7 @@ |
165 | indicate the eggs from site-packages that have been selected. You'll see |
166 | we have two: demo 0.3 and demoneeded 1.1. |
167 | |
168 | - >>> print system(py_path+" "+buildout+" -v") |
169 | + >>> print system(buildout+" -v") |
170 | Installing 'zc.buildout', 'setuptools'. |
171 | We have a develop egg: zc.buildout V |
172 | We have the best distribution that satisfies 'setuptools'. |
173 | @@ -2254,6 +2254,94 @@ |
174 | |
175 | """ |
176 | |
177 | +def bootstrap_makes_buildout_that_works_with_system_python(): |
178 | + """ |
179 | +In order to work smoothly with a system Python, bootstrapping creates |
180 | +the buildout script with |
181 | +zc.buildout.easy_install.sitepackage_safe_scripts. If it did not, a |
182 | +variety of problems might happen. For instance, if another version of |
183 | +buildout or setuptools is installed in the site-packages than is |
184 | +desired, it may cause a problem. |
185 | + |
186 | +A problem actually experienced in the field is when |
187 | +a recipe wants a different version of a dependency that is installed in |
188 | +site-packages. We will create a similar situation, and show that it is now |
189 | +handled. |
190 | + |
191 | +First let's write a dummy recipe. |
192 | + |
193 | + >>> mkdir(sample_buildout, 'recipes') |
194 | + >>> write(sample_buildout, 'recipes', 'dummy.py', |
195 | + ... ''' |
196 | + ... import logging, os, zc.buildout |
197 | + ... |
198 | + ... class Dummy: |
199 | + ... |
200 | + ... def __init__(self, buildout, name, options): |
201 | + ... pass |
202 | + ... |
203 | + ... def install(self): |
204 | + ... return () |
205 | + ... |
206 | + ... def update(self): |
207 | + ... pass |
208 | + ... ''') |
209 | + >>> write(sample_buildout, 'recipes', 'setup.py', |
210 | + ... ''' |
211 | + ... from setuptools import setup |
212 | + ... |
213 | + ... setup( |
214 | + ... name = "recipes", |
215 | + ... entry_points = {'zc.buildout': ['dummy = dummy:Dummy']}, |
216 | + ... install_requires = 'demoneeded==1.2c1', |
217 | + ... ) |
218 | + ... ''') |
219 | + >>> write(sample_buildout, 'recipes', 'README.txt', " ") |
220 | + |
221 | +Now we'll try to use it with a Python that has a different version of |
222 | +demoneeded installed. |
223 | + |
224 | + >>> py_path, site_packages_path = make_py() |
225 | + >>> create_sample_sys_install(site_packages_path) |
226 | + >>> rmdir('develop-eggs') |
227 | + >>> from zc.buildout.testing import make_buildout |
228 | + >>> make_buildout(executable=py_path) |
229 | + >>> write(sample_buildout, 'buildout.cfg', |
230 | + ... ''' |
231 | + ... [buildout] |
232 | + ... develop = recipes |
233 | + ... parts = dummy |
234 | + ... find-links = %(link_server)s |
235 | + ... executable = %(py_path)s |
236 | + ... |
237 | + ... [dummy] |
238 | + ... recipe = recipes:dummy |
239 | + ... ''' % globals()) |
240 | + |
241 | +Now we actually run the buildout. Before the change, we got the following |
242 | +error: |
243 | + |
244 | + Develop: '/sample-buildout/recipes' |
245 | + While: |
246 | + Installing. |
247 | + Getting section dummy. |
248 | + Initializing section dummy. |
249 | + Installing recipe recipes. |
250 | + Error: There is a version conflict. |
251 | + We already have: demoneeded 1.1 |
252 | + but recipes 0.0.0 requires 'demoneeded==1.2c1'. |
253 | + |
254 | +Now, it is handled smoothly. |
255 | + |
256 | + >>> print system(buildout) |
257 | + Develop: '/sample-buildout/recipes' |
258 | + Getting distribution for 'demoneeded==1.2c1'. |
259 | + Got demoneeded 1.2c1. |
260 | + Installing dummy. |
261 | + <BLANKLINE> |
262 | + |
263 | + """ |
264 | + |
265 | if sys.version_info > (2, 4): |
266 | def test_exit_codes(): |
267 | """ |
268 | @@ -3367,7 +3455,8 @@ |
269 | here = os.getcwd() |
270 | os.chdir(os.path.dirname(dist.location)) |
271 | assert os.spawnle( |
272 | - os.P_WAIT, sys.executable, zc.buildout.easy_install._safe_arg (sys.executable), |
273 | + os.P_WAIT, sys.executable, |
274 | + zc.buildout.easy_install._safe_arg(sys.executable), |
275 | os.path.join(os.path.dirname(dist.location), 'setup.py'), |
276 | '-q', 'bdist_egg', '-d', eggs, |
277 | dict(os.environ, |
278 | |
279 | === modified file 'z3c.recipe.scripts_/src/z3c/recipe/scripts/README.txt' |
280 | --- z3c.recipe.scripts_/src/z3c/recipe/scripts/README.txt 2010-02-24 23:23:14 +0000 |
281 | +++ z3c.recipe.scripts_/src/z3c/recipe/scripts/README.txt 2010-02-24 23:23:14 +0000 |
282 | @@ -39,6 +39,35 @@ |
283 | from your eggs. See the section on this option for motivations and |
284 | warnings. |
285 | |
286 | +allowed-eggs-from-site-packages |
287 | + Sometimes you need or want to control what eggs from site-packages are |
288 | + used. The allowed-eggs-from-site-packages option allows you to specify a |
289 | + whitelist of project names that may be included from site-packages. You |
290 | + can use globs to specify the value. It defaults to a single value of '*', |
291 | + indicating that any package may come from site-packages. |
292 | + |
293 | + Here's a usage example:: |
294 | + |
295 | + [buildout] |
296 | + ... |
297 | + |
298 | + allowed-eggs-from-site-packages = |
299 | + demo |
300 | + bigdemo |
301 | + zope.* |
302 | + |
303 | + This option interacts with the ``include-site-packages`` option in the |
304 | + following ways. |
305 | + |
306 | + If ``include-site-packages`` is true, then |
307 | + ``allowed-eggs-from-site-packages`` filters what eggs from site-packages |
308 | + may be chosen. Therefore, if ``allowed-eggs-from-site-packages`` is an |
309 | + empty list, then no eggs from site-packages are chosen, but site-packages |
310 | + will still be included at the end of path lists. |
311 | + |
312 | + If ``include-site-packages`` is false, the value of |
313 | + ``allowed-eggs-from-site-packages`` is irrelevant. |
314 | + |
315 | extends |
316 | You can extend another section using this value. It is intended to be |
317 | used by extending a section that uses this package's scripts recipe. |
318 | @@ -215,7 +244,8 @@ |
319 | >>> print site_packages_path |
320 | /executable_buildout/site-packages |
321 | |
322 | -Now let's take a look at include-site-packages. |
323 | +Now let's take a look at include-site-packages. The default is false, |
324 | +so we will set it to true. |
325 | |
326 | >>> write(sample_buildout, 'buildout.cfg', |
327 | ... """ |
328 | @@ -236,6 +266,8 @@ |
329 | Installing py. |
330 | Generated interpreter '/sample-buildout/bin/py'. |
331 | |
332 | +Now executable_buildout/site-packages is included in sys.path. |
333 | + |
334 | >>> print system(join(sample_buildout, 'bin', 'py') + |
335 | ... ''' -c "import sys, pprint; pprint.pprint(sys.path)"''') |
336 | ... # doctest: +ELLIPSIS |
337 | @@ -248,6 +280,96 @@ |
338 | '/executable_buildout/site-packages'] |
339 | <BLANKLINE> |
340 | |
341 | +As described above, the allowed-eggs-from-site-packages option lets us |
342 | +control what site-packages eggs zc.buildout will allow to fulfill |
343 | +dependencies. The behavior was described above with an example (and the |
344 | +implementation is tested elsewhere), so we'll only look at some simple and |
345 | +common use cases here. |
346 | + |
347 | +Sometimes you may want to allow site-packages to be available but you don't |
348 | +want your package to depend on it using setup.py. For instance, perhaps you |
349 | +are writing an application, and you want to depend on your system's packaging |
350 | +of the PostgreSQL code, but the system Python does not use eggs to |
351 | +package it, so you need to manage the two separately. In this case, you |
352 | +might not want to use any eggs from site-packages, but you want it available. |
353 | +In this case, you can use allowed-eggs-from-site-packages with an empty value |
354 | +to keep any egg from being used from site-packages. |
355 | + |
356 | +Here's an example. Let's say we have a Python with demo and demoneeded |
357 | +installed as eggs in the system Python. Normally, they will be used to |
358 | +fulfill dependencies, because allowed-eggs-from-site-packages defaults to |
359 | +the value "*" (allow any package). (We use an empty find-links value to say |
360 | +that buildout may not look elsewhere for the package. We use a different |
361 | +eggs-directory for isolation, so that eggs obtained other parts of the |
362 | +document do not affect this example.) |
363 | + |
364 | + >>> from zc.buildout.tests import create_sample_sys_install |
365 | + >>> create_sample_sys_install(site_packages_path) |
366 | + >>> import zc.buildout.easy_install |
367 | + >>> zc.buildout.easy_install.clear_index_cache() |
368 | + |
369 | + >>> write('buildout.cfg', |
370 | + ... ''' |
371 | + ... [buildout] |
372 | + ... parts = eggs |
373 | + ... eggs-directory = tmpeggs |
374 | + ... find-links = |
375 | + ... |
376 | + ... [primed_python] |
377 | + ... executable = %(py_path)s |
378 | + ... |
379 | + ... [eggs] |
380 | + ... recipe = z3c.recipe.scripts |
381 | + ... include-site-packages = true |
382 | + ... python = primed_python |
383 | + ... eggs = demoneeded |
384 | + ... ''' % globals()) |
385 | + |
386 | + >>> print system(buildout) |
387 | + Creating directory '/sample-buildout/tmpeggs'. |
388 | + Uninstalling py. |
389 | + Installing eggs. |
390 | + <BLANKLINE> |
391 | + |
392 | +That succeeds fine, getting demoneeded from the Python site-packages. |
393 | + |
394 | +However, when allowed-eggs-from-site-packages is an empty value, demoneeded |
395 | +is not allowed to come from site-packages, and the buildout fails. |
396 | + |
397 | + >>> zc.buildout.easy_install.clear_index_cache() |
398 | + >>> rmdir('tmpeggs') |
399 | + >>> write('buildout.cfg', |
400 | + ... ''' |
401 | + ... [buildout] |
402 | + ... parts = eggs |
403 | + ... eggs-directory = tmpeggs |
404 | + ... find-links = |
405 | + ... |
406 | + ... [primed_python] |
407 | + ... executable = %(py_path)s |
408 | + ... |
409 | + ... [eggs] |
410 | + ... recipe = z3c.recipe.scripts |
411 | + ... include-site-packages = true |
412 | + ... allowed-eggs-from-site-packages = |
413 | + ... eggs = demoneeded |
414 | + ... ''' % globals()) |
415 | + >>> print system(buildout) |
416 | + Creating directory '/sample-buildout/tmpeggs'. |
417 | + Uninstalling eggs. |
418 | + Installing eggs. |
419 | + Couldn't find index page for 'demoneeded' (maybe misspelled?) |
420 | + Getting distribution for 'demoneeded'. |
421 | + While: |
422 | + Installing eggs. |
423 | + Getting distribution for 'demoneeded'. |
424 | + Error: Couldn't find a distribution for 'demoneeded'. |
425 | + <BLANKLINE> |
426 | + |
427 | +Remember that you can provide multiple lines to the |
428 | +allowed-eggs-from-site-packages option, each specifying a whitelist of |
429 | +allowed packages. Globs (* and ?) are allowed. |
430 | + |
431 | Next we will use the exec-sitecustomize option. It simply copies |
432 | Python's underlying sitecustomize module, if it exists, to the local |
433 | version. The os.environ change shown above in the make_py call will go |
434 | @@ -268,7 +390,6 @@ |
435 | ... """ % dict(server=link_server, py_path=py_path)) |
436 | |
437 | >>> print system(buildout), |
438 | - Uninstalling py. |
439 | Installing py. |
440 | Generated interpreter '/sample-buildout/bin/py'. |
441 | |
442 | @@ -289,7 +410,8 @@ |
443 | Options |
444 | ------- |
445 | |
446 | -We'll focus now on the options that are different than zc.recipe.egg. |
447 | +We'll focus now on the remaining options that are different than |
448 | +zc.recipe.egg. |
449 | |
450 | Let's look at the ``extends`` option first. |
451 | |
452 | @@ -337,6 +459,7 @@ |
453 | been created. |
454 | |
455 | >>> ls(sample_buildout, 'parts') |
456 | + d buildout |
457 | d demo |
458 | d python |
459 | |
460 | |
461 | === modified file 'z3c.recipe.scripts_/src/z3c/recipe/scripts/scripts.py' |
462 | --- z3c.recipe.scripts_/src/z3c/recipe/scripts/scripts.py 2010-02-24 23:23:14 +0000 |
463 | +++ z3c.recipe.scripts_/src/z3c/recipe/scripts/scripts.py 2010-02-24 23:23:14 +0000 |
464 | @@ -31,6 +31,11 @@ |
465 | b_options['parts-directory'], self.name) |
466 | |
467 | value = options.setdefault( |
468 | + 'allowed-eggs-from-site-packages', |
469 | + '*') |
470 | + self.allowed_eggs = tuple(name.strip() for name in value.split('\n')) |
471 | + |
472 | + value = options.setdefault( |
473 | 'include-site-packages', |
474 | b_options.get('include-site-packages', 'false')) |
475 | if value not in ('true', 'false'): |
476 | |
477 | === modified file 'z3c.recipe.scripts_/src/z3c/recipe/scripts/tests.py' |
478 | --- z3c.recipe.scripts_/src/z3c/recipe/scripts/tests.py 2010-02-24 23:23:14 +0000 |
479 | +++ z3c.recipe.scripts_/src/z3c/recipe/scripts/tests.py 2010-02-24 23:23:14 +0000 |
480 | @@ -235,10 +235,171 @@ |
481 | '/foo/bar', |
482 | join(base, 'spam') |
483 | ]... |
484 | - |
485 | - |
486 | """ |
487 | |
488 | +def include_site_packages_option_reusing_eggs(): |
489 | + """ |
490 | +The include-site-packages buildout option not only controls whether |
491 | +site-packages are included in the path, but whether eggs in site-packages |
492 | +can be used to fulfill direct and indirect dependencies of your package. If |
493 | +it did not, it might fail to exclude site-packages because one of the |
494 | +dependencies actually was supposed to be fulfilled with it. |
495 | + |
496 | +The default is ``include-site-packages = false``. This makes it possible to |
497 | +easily use a system Python. As a demonstration, we will start with a |
498 | +Python executable that has the "demoneeded" and "demo" eggs installed. |
499 | +The eggs are not found. |
500 | + |
501 | + >>> from zc.buildout.tests import create_sample_sys_install |
502 | + >>> py_path, site_packages_path = make_py() |
503 | + >>> create_sample_sys_install(site_packages_path) |
504 | + >>> write('buildout.cfg', |
505 | + ... ''' |
506 | + ... [buildout] |
507 | + ... parts = eggs |
508 | + ... find-links = |
509 | + ... |
510 | + ... [primed_python] |
511 | + ... executable = %(py_path)s |
512 | + ... |
513 | + ... [eggs] |
514 | + ... recipe = z3c.recipe.scripts |
515 | + ... python = primed_python |
516 | + ... eggs = demoneeded |
517 | + ... ''' % globals()) |
518 | + >>> print system(buildout) |
519 | + Installing eggs. |
520 | + Couldn't find index page for 'demoneeded' (maybe misspelled?) |
521 | + Getting distribution for 'demoneeded'. |
522 | + While: |
523 | + Installing eggs. |
524 | + Getting distribution for 'demoneeded'. |
525 | + Error: Couldn't find a distribution for 'demoneeded'. |
526 | + <BLANKLINE> |
527 | + |
528 | +However, if we set include-site-packages to true, the package will be found. |
529 | +Notice we do not set find-links, but the eggs are still found because |
530 | +they are in the executable's path. |
531 | + |
532 | + >>> write('buildout.cfg', |
533 | + ... ''' |
534 | + ... [buildout] |
535 | + ... parts = eggs |
536 | + ... find-links = |
537 | + ... |
538 | + ... [primed_python] |
539 | + ... executable = %(py_path)s |
540 | + ... |
541 | + ... [eggs] |
542 | + ... recipe = z3c.recipe.scripts |
543 | + ... python = primed_python |
544 | + ... include-site-packages = true |
545 | + ... eggs = demoneeded |
546 | + ... ''' % globals()) |
547 | + |
548 | + >>> print system(buildout) |
549 | + Installing eggs. |
550 | + <BLANKLINE> |
551 | + |
552 | +We get an error if we specify anything but true or false: |
553 | + |
554 | + >>> write('buildout.cfg', |
555 | + ... ''' |
556 | + ... [buildout] |
557 | + ... parts = eggs |
558 | + ... find-links = %(link_server)s |
559 | + ... |
560 | + ... [eggs] |
561 | + ... recipe = z3c.recipe.scripts |
562 | + ... include-site-packages = no |
563 | + ... eggs = other |
564 | + ... ''' % globals()) |
565 | + |
566 | + >>> print system(buildout) |
567 | + While: |
568 | + Installing. |
569 | + Getting section eggs. |
570 | + Initializing part eggs. |
571 | + Error: Invalid value for include-site-packages option: no |
572 | + <BLANKLINE> |
573 | + |
574 | + """ |
575 | + |
576 | +def allowed_eggs_from_site_packages_option(): |
577 | + """ |
578 | +The allowed-eggs-from-site-packages option allows you to specify a |
579 | +whitelist of project names that may be included from site-packages. |
580 | + |
581 | +In the test below, our "py_path" has the "demoneeded" and "demo" |
582 | +packages available. We'll simply be asking for "demoneeded" here. The |
583 | +default value of '*' will allow it, as we've seen elsewhere. Here we |
584 | +explicitly use a "*" for the same result. This also shows that we |
585 | +correctly parse a single-line value. |
586 | + |
587 | + |
588 | + >>> from zc.buildout.tests import create_sample_sys_install |
589 | + >>> py_path, site_packages_path = make_py() |
590 | + >>> create_sample_sys_install(site_packages_path) |
591 | + >>> write('buildout.cfg', |
592 | + ... ''' |
593 | + ... [buildout] |
594 | + ... parts = eggs |
595 | + ... find-links = |
596 | + ... |
597 | + ... [primed_python] |
598 | + ... executable = %(py_path)s |
599 | + ... |
600 | + ... [eggs] |
601 | + ... recipe = z3c.recipe.scripts |
602 | + ... include-site-packages = true |
603 | + ... allowed-eggs-from-site-packages = * |
604 | + ... python = primed_python |
605 | + ... eggs = demoneeded |
606 | + ... ''' % globals()) |
607 | + |
608 | + >>> print system(buildout) |
609 | + Installing eggs. |
610 | + <BLANKLINE> |
611 | + |
612 | +Specifying the egg exactly will work as well. This shows we correctly |
613 | +parse a multi-line value. |
614 | + |
615 | + >>> zc.buildout.easy_install.clear_index_cache() |
616 | + >>> write('buildout.cfg', |
617 | + ... ''' |
618 | + ... [buildout] |
619 | + ... parts = eggs |
620 | + ... find-links = |
621 | + ... |
622 | + ... [primed_python] |
623 | + ... executable = %(py_path)s |
624 | + ... |
625 | + ... [eggs] |
626 | + ... recipe = z3c.recipe.scripts |
627 | + ... include-site-packages = true |
628 | + ... allowed-eggs-from-site-packages = other |
629 | + ... demoneeded |
630 | + ... python = primed_python |
631 | + ... eggs = demoneeded |
632 | + ... ''' % globals()) |
633 | + |
634 | + >>> print system(buildout) |
635 | + Uninstalling eggs. |
636 | + Installing eggs. |
637 | + <BLANKLINE> |
638 | + |
639 | +It will also work if we use a glob ("*" or "?"). (We won't show that here |
640 | +because we already tested it in |
641 | +zc.buildout.tests.allowed_eggs_from_site_packages.) |
642 | + |
643 | +However, if we do not include "demoneeded" in the |
644 | +"allowed-eggs-from-site-packages" key, we get an error, because the |
645 | +packages are not available in any links, and they are not allowed to |
646 | +come from the executable's site packages. (We won't show that here |
647 | +because we already tested it in the same test mentioned above.) |
648 | + |
649 | + """ |
650 | + |
651 | def setUp(test): |
652 | zc.buildout.tests.easy_install_SetUp(test) |
653 | zc.buildout.testing.install_develop('zc.recipe.egg', test) |
654 | |
655 | === modified file 'zc.recipe.egg_/src/zc/recipe/egg/egg.py' |
656 | --- zc.recipe.egg_/src/zc/recipe/egg/egg.py 2010-02-24 23:23:14 +0000 |
657 | +++ zc.recipe.egg_/src/zc/recipe/egg/egg.py 2010-02-24 23:23:14 +0000 |
658 | @@ -22,6 +22,8 @@ |
659 | |
660 | class Eggs(object): |
661 | |
662 | + include_site_packages = allowed_eggs = None |
663 | + |
664 | def __init__(self, buildout, name, options): |
665 | self.buildout = buildout |
666 | self.name = self.default_eggs = name |
667 | @@ -76,6 +78,8 @@ |
668 | distributions, options['executable'], |
669 | [options['develop-eggs-directory'], |
670 | options['eggs-directory']], |
671 | + include_site_packages=self.include_site_packages, |
672 | + allowed_eggs_from_site_packages=self.allowed_eggs, |
673 | ) |
674 | else: |
675 | kw = {} |
676 | @@ -88,6 +92,8 @@ |
677 | executable=options['executable'], |
678 | path=[options['develop-eggs-directory']], |
679 | newest=b_options.get('newest') == 'true', |
680 | + include_site_packages=self.include_site_packages, |
681 | + allowed_eggs_from_site_packages=self.allowed_eggs, |
682 | allow_hosts=self.allow_hosts, |
683 | **kw) |
684 |
This branch hooks up the code introduced in ~gary/zc. buildout/ python- support- 5-initial- egg-control and thus fixes a significant problem for people trying to use this branch without allowing site-packages into the buildout.
It also is intended to be the final branch in the series, so I ran tests again on Python 2.4, 2.5, and 2.6, and on Windows. The Windows test had some failures (in the test infrastructure, not in the core code), so this branch fixes some of those failures as well (the ``do_build`` code and the change to dev.py).
Thank you
Gary