Merge lp:~gary/zc.buildout/python-support-5-initial-egg-control into lp:zc.buildout
- python-support-5-initial-egg-control
- Merge into trunk
Status: | Needs review |
---|---|
Proposed branch: | lp:~gary/zc.buildout/python-support-5-initial-egg-control |
Merge into: | lp:zc.buildout |
Prerequisite: | lp:~gary/zc.buildout/python-support-4 |
Diff against target: |
1188 lines (+651/-156) 8 files modified
.bzrignore (+9/-0) src/zc/buildout/easy_install.py (+223/-104) src/zc/buildout/easy_install.txt (+73/-3) src/zc/buildout/testing.py (+33/-6) src/zc/buildout/tests.py (+301/-31) z3c.recipe.scripts_/src/z3c/recipe/scripts/README.txt (+5/-5) z3c.recipe.scripts_/src/z3c/recipe/scripts/scripts.py (+6/-6) z3c.recipe.scripts_/src/z3c/recipe/scripts/tests.py (+1/-1) |
To merge this branch: | bzr merge lp:~gary/zc.buildout/python-support-5-initial-egg-control |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Francis J. Lacoste (community) | Approve | ||
Review via email: mp+20010@code.launchpad.net |
Commit message
Description of the change
Gary Poster (gary) wrote : | # |
Note also that I moved _get_system_paths to the top of easy_install.py because it was now used by both the Installer and sitepackage_
Francis J. Lacoste (flacoste) wrote : | # |
Hi Gary,
My only comment is about some API incompatible changes you made. Not sure it's a problem or not. Otherwise, all fine.
> === modified file 'src/zc/
> def use_dependency_
> old = Installer.
> if setting is not None:
> @@ -858,9 +1014,13 @@
> links=(), index=None,
> executable=
> path=None, working_set=None, newest=True, versions=None,
> - use_dependency_
> + use_dependency_
> + allowed_
> installer = Installer(dest, links, index, executable, always_unzip, path,
> newest, versions, use_dependency_
> + include_
> + allowed_
> + allowed_
> allow_hosts=
> return installer.
>
> @@ -868,9 +1028,14 @@
> def build(spec, dest, build_ext,
> links=(), index=None,
> executable=
> - path=None, newest=True, versions=None, allow_hosts=
> + path=None, newest=True, versions=None, include_
> + allowed_
> installer = Installer(dest, links, index, executable, True, path, newest,
> - versions, allow_hosts=
> + versions,
> + include_
> + allowed_
> + allowed_
> + allow_hosts=
> return installer.
>
Should we care about API compatibility in these two functions? You added
parameters before allow_hosts. Theorically, this breaks API compatibility for
call sites passing things by parameters. Arguably a bad idea with such a
signature, but it's your call :-)
- 555. By Gary Poster
-
be a better backwards-
compatibilty citizen
Gary Poster (gary) wrote : | # |
Yeah, what I did was on purpose and after consideration, but maybe wrong all the same. :-) I changed it.
Thanks!
Unmerged revisions
- 555. By Gary Poster
-
be a better backwards-
compatibilty citizen - 554. By Gary Poster
-
merge from gary-4
- 553. By Gary Poster
-
merge from gary-4
- 552. By Gary Poster
-
settle on "include-
site-packages" and migrate all variations of "add-site-packages" to that spelling. - 551. By Gary Poster
-
merge from gary-4
- 550. By Gary Poster
-
support limiting packages from site-packages
- 549. By Gary Poster
-
propagate merge from gary-3 <- gary-2 <- gary-1 <- trunk
- 548. By gary
-
fix some tests on other Python versions
- 547. By gary
-
revert attempt to skip some of the pkg_resources dance: it caused me trouble.
- 546. By gary
-
simplify resulting code a bit more and try again to remove warnings
Preview Diff
1 | === added file '.bzrignore' |
2 | --- .bzrignore 1970-01-01 00:00:00 +0000 |
3 | +++ .bzrignore 2010-02-24 23:22:13 +0000 |
4 | @@ -0,0 +1,9 @@ |
5 | +.installed.cfg |
6 | +bin |
7 | +build |
8 | +develop-eggs |
9 | +eggs |
10 | +parts |
11 | +src/zc.buildout.egg-info |
12 | +z3c.recipe.scripts_/src/z3c.recipe.scripts.egg-info |
13 | +zc.recipe.egg_/src/zc.recipe.egg.egg-info |
14 | |
15 | === modified file 'src/zc/buildout/easy_install.py' |
16 | --- src/zc/buildout/easy_install.py 2010-02-24 23:22:13 +0000 |
17 | +++ src/zc/buildout/easy_install.py 2010-02-24 23:22:13 +0000 |
18 | @@ -19,6 +19,7 @@ |
19 | """ |
20 | |
21 | import distutils.errors |
22 | +import fnmatch |
23 | import glob |
24 | import logging |
25 | import os |
26 | @@ -68,6 +69,64 @@ |
27 | if os.path.normpath(setuptools_loc) != os.path.normpath(buildout_loc): |
28 | buildout_and_setuptools_path.append(buildout_loc) |
29 | |
30 | +def _get_system_paths(executable): |
31 | + """Return lists of standard lib and site paths for executable. |
32 | + """ |
33 | + # We want to get a list of the site packages, which is not easy. |
34 | + # The canonical way to do this is to use |
35 | + # distutils.sysconfig.get_python_lib(), but that only returns a |
36 | + # single path, which does not reflect reality for many system |
37 | + # Pythons, which have multiple additions. Instead, we start Python |
38 | + # with -S, which does not import site.py and set up the extra paths |
39 | + # like site-packages or (Ubuntu/Debian) dist-packages and |
40 | + # python-support. We then compare that sys.path with the normal one |
41 | + # (minus user packages if this is Python 2.6, because we don't |
42 | + # support those (yet?). The set of the normal one minus the set of |
43 | + # the ones in ``python -S`` is the set of packages that are |
44 | + # effectively site-packages. |
45 | + # |
46 | + # The given executable might not be the current executable, so it is |
47 | + # appropriate to do another subprocess to figure out what the |
48 | + # additional site-package paths are. Moreover, even if this |
49 | + # executable *is* the current executable, this code might be run in |
50 | + # the context of code that has manipulated the sys.path--for |
51 | + # instance, to add local zc.buildout or setuptools eggs. |
52 | + def get_sys_path(*args, **kwargs): |
53 | + cmd = [executable] |
54 | + cmd.extend(args) |
55 | + cmd.extend([ |
56 | + "-c", "import sys, os;" |
57 | + "print repr([os.path.normpath(p) for p in sys.path if p])"]) |
58 | + # Windows needs some (as yet to be determined) part of the real env. |
59 | + env = os.environ.copy() |
60 | + env.update(kwargs) |
61 | + _proc = subprocess.Popen( |
62 | + cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env) |
63 | + stdout, stderr = _proc.communicate(); |
64 | + if _proc.returncode: |
65 | + raise RuntimeError( |
66 | + 'error trying to get system packages:\n%s' % (stderr,)) |
67 | + res = eval(stdout.strip()) |
68 | + try: |
69 | + res.remove('.') |
70 | + except ValueError: |
71 | + pass |
72 | + return res |
73 | + stdlib = get_sys_path('-S') # stdlib only |
74 | + no_user_paths = get_sys_path(PYTHONNOUSERSITE='x') |
75 | + site_paths = [p for p in no_user_paths if p not in stdlib] |
76 | + return (stdlib, site_paths) |
77 | + |
78 | +def _get_version_info(executable): |
79 | + cmd = [executable, '-Sc', 'import sys; print repr(sys.version_info)'] |
80 | + _proc = subprocess.Popen( |
81 | + cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE) |
82 | + stdout, stderr = _proc.communicate(); |
83 | + if _proc.returncode: |
84 | + raise RuntimeError( |
85 | + 'error trying to get system packages:\n%s' % (stderr,)) |
86 | + return eval(stdout.strip()) |
87 | + |
88 | |
89 | class IncompatibleVersionError(zc.buildout.UserError): |
90 | """A specified version is incompatible with a given requirement. |
91 | @@ -110,7 +169,12 @@ |
92 | |
93 | |
94 | _indexes = {} |
95 | -def _get_index(executable, index_url, find_links, allow_hosts=('*',)): |
96 | +def _get_index(executable, index_url, find_links, allow_hosts=('*',), |
97 | + path=None): |
98 | + # If path is None, the index will use sys.path. If you provide an empty |
99 | + # path ([]), it will complain uselessly about missing index pages for |
100 | + # packages found in the paths that you expect to use. Therefore, this path |
101 | + # is always the same as the _env path in the Installer. |
102 | key = executable, index_url, tuple(find_links) |
103 | index = _indexes.get(key) |
104 | if index is not None: |
105 | @@ -119,7 +183,8 @@ |
106 | if index_url is None: |
107 | index_url = default_index_url |
108 | index = AllowHostsPackageIndex( |
109 | - index_url, hosts=allow_hosts, python=_get_version(executable) |
110 | + index_url, hosts=allow_hosts, search_path=path, |
111 | + python=_get_version(executable) |
112 | ) |
113 | |
114 | if find_links: |
115 | @@ -216,6 +281,8 @@ |
116 | _use_dependency_links = True |
117 | _allow_picked_versions = True |
118 | _always_unzip = False |
119 | + _include_site_packages = True |
120 | + _allowed_eggs_from_site_packages = ('*',) |
121 | |
122 | def __init__(self, |
123 | dest=None, |
124 | @@ -227,7 +294,9 @@ |
125 | newest=True, |
126 | versions=None, |
127 | use_dependency_links=None, |
128 | - allow_hosts=('*',) |
129 | + allow_hosts=('*',), |
130 | + include_site_packages=None, |
131 | + allowed_eggs_from_site_packages=None |
132 | ): |
133 | self._dest = dest |
134 | self._allow_hosts = allow_hosts |
135 | @@ -249,7 +318,28 @@ |
136 | self._executable = executable |
137 | if always_unzip is not None: |
138 | self._always_unzip = always_unzip |
139 | - path = (path and path[:] or []) + buildout_and_setuptools_path |
140 | + path = (path and path[:] or []) |
141 | + if include_site_packages is not None: |
142 | + self._include_site_packages = include_site_packages |
143 | + if allowed_eggs_from_site_packages is not None: |
144 | + self._allowed_eggs_from_site_packages = tuple( |
145 | + allowed_eggs_from_site_packages) |
146 | + stdlib, self._site_packages = _get_system_paths(executable) |
147 | + version_info = _get_version_info(executable) |
148 | + if version_info == sys.version_info: |
149 | + # Maybe we can add the buildout and setuptools path. If we |
150 | + # are including site_packages, we only have to include the extra |
151 | + # bits here, so we don't duplicate. On the other hand, if we |
152 | + # are not including site_packages, we only want to include the |
153 | + # parts that are not in site_packages, so the code is the same. |
154 | + path.extend( |
155 | + set(buildout_and_setuptools_path).difference( |
156 | + self._site_packages)) |
157 | + if self._include_site_packages: |
158 | + path.extend(self._site_packages) |
159 | + # else we could try to still include the buildout_and_setuptools_path |
160 | + # if the elements are not in site_packages, but we're not bothering |
161 | + # with this optimization for now, in the name of code simplicity. |
162 | if dest is not None and dest not in path: |
163 | path.insert(0, dest) |
164 | self._path = path |
165 | @@ -258,13 +348,42 @@ |
166 | self._newest = newest |
167 | self._env = pkg_resources.Environment(path, |
168 | python=_get_version(executable)) |
169 | - self._index = _get_index(executable, index, links, self._allow_hosts) |
170 | + self._index = _get_index(executable, index, links, self._allow_hosts, |
171 | + self._path) |
172 | |
173 | if versions is not None: |
174 | self._versions = versions |
175 | |
176 | + _allowed_eggs_from_site_packages_regex = None |
177 | + def allow_site_package_egg(self, name): |
178 | + if (not self._include_site_packages or |
179 | + not self._allowed_eggs_from_site_packages): |
180 | + # If the answer is a blanket "no," perform a shortcut. |
181 | + return False |
182 | + if self._allowed_eggs_from_site_packages_regex is None: |
183 | + pattern = '(%s)' % ( |
184 | + '|'.join( |
185 | + fnmatch.translate(name) |
186 | + for name in self._allowed_eggs_from_site_packages), |
187 | + ) |
188 | + self._allowed_eggs_from_site_packages_regex = re.compile(pattern) |
189 | + return bool(self._allowed_eggs_from_site_packages_regex.match(name)) |
190 | + |
191 | def _satisfied(self, req, source=None): |
192 | - dists = [dist for dist in self._env[req.project_name] if dist in req] |
193 | + # We get all distributions that match the given requirement. If we are |
194 | + # not supposed to include site-packages for the given egg, we also |
195 | + # filter those out. Even if include_site_packages is False and so we |
196 | + # have excluded site packages from the _env's paths (see |
197 | + # Installer.__init__), we need to do the filtering here because an |
198 | + # .egg-link, such as one for setuptools or zc.buildout installed by |
199 | + # zc.buildout.buildout.Buildout.bootstrap, can indirectly include a |
200 | + # path in our _site_packages. |
201 | + dists = [dist for dist in self._env[req.project_name] if ( |
202 | + dist in req and ( |
203 | + dist.location not in self._site_packages or |
204 | + self.allow_site_package_egg(dist.project_name)) |
205 | + ) |
206 | + ] |
207 | if not dists: |
208 | logger.debug('We have no distributions for %s that satisfies %r.', |
209 | req.project_name, str(req)) |
210 | @@ -465,14 +584,22 @@ |
211 | # Nothing is available. |
212 | return None |
213 | |
214 | - # Filter the available dists for the requirement and source flag |
215 | - dists = [dist for dist in index[requirement.project_name] |
216 | - if ((dist in requirement) |
217 | - and |
218 | - ((not source) or |
219 | - (dist.precedence == pkg_resources.SOURCE_DIST) |
220 | - ) |
221 | - ) |
222 | + # Filter the available dists for the requirement and source flag. If |
223 | + # we are not supposed to include site-packages for the given egg, we |
224 | + # also filter those out. Even if include_site_packages is False and so |
225 | + # we have excluded site packages from the _env's paths (see |
226 | + # Installer.__init__), we need to do the filtering here because an |
227 | + # .egg-link, such as one for setuptools or zc.buildout installed by |
228 | + # zc.buildout.buildout.Buildout.bootstrap, can indirectly include a |
229 | + # path in our _site_packages. |
230 | + dists = [dist for dist in index[requirement.project_name] if ( |
231 | + dist in requirement and ( |
232 | + dist.location not in self._site_packages or |
233 | + self.allow_site_package_egg(dist.project_name)) |
234 | + and ( |
235 | + (not source) or |
236 | + (dist.precedence == pkg_resources.SOURCE_DIST)) |
237 | + ) |
238 | ] |
239 | |
240 | # If we prefer final dists, filter for final and use the |
241 | @@ -632,7 +759,7 @@ |
242 | self._links.append(link) |
243 | self._index = _get_index(self._executable, |
244 | self._index_url, self._links, |
245 | - self._allow_hosts) |
246 | + self._allow_hosts, self._path) |
247 | |
248 | for dist in dists: |
249 | # Check whether we picked a version and, if we did, report it: |
250 | @@ -713,35 +840,52 @@ |
251 | self._maybe_add_setuptools(ws, dist) |
252 | |
253 | # OK, we have the requested distributions and they're in the working |
254 | - # set, but they may have unmet requirements. We'll simply keep |
255 | - # trying to resolve requirements, adding missing requirements as they |
256 | - # are reported. |
257 | - # |
258 | - # Note that we don't pass in the environment, because we want |
259 | + # set, but they may have unmet requirements. We'll resolve these |
260 | + # requirements. This is code modified from |
261 | + # pkg_resources.WorkingSet.resolve. We can't reuse that code directly |
262 | + # because we have to constrain our requirements (see |
263 | + # versions_section_ignored_for_dependency_in_favor_of_site_packages in |
264 | + # zc.buildout.tests). |
265 | + requirements.reverse() # Set up the stack. |
266 | + processed = {} # This is a set of processed requirements. |
267 | + best = {} # This is a mapping of key -> dist. |
268 | + # Note that we don't use the existing environment, because we want |
269 | # to look for new eggs unless what we have is the best that |
270 | # matches the requirement. |
271 | - while 1: |
272 | - try: |
273 | - ws.resolve(requirements) |
274 | - except pkg_resources.DistributionNotFound, err: |
275 | - [requirement] = err |
276 | - requirement = self._constrain(requirement) |
277 | - if destination: |
278 | - logger.debug('Getting required %r', str(requirement)) |
279 | - else: |
280 | - logger.debug('Adding required %r', str(requirement)) |
281 | - _log_requirement(ws, requirement) |
282 | - |
283 | - for dist in self._get_dist(requirement, ws, self._always_unzip |
284 | - ): |
285 | - |
286 | - ws.add(dist) |
287 | - self._maybe_add_setuptools(ws, dist) |
288 | - except pkg_resources.VersionConflict, err: |
289 | - raise VersionConflict(err, ws) |
290 | - else: |
291 | - break |
292 | - |
293 | + env = pkg_resources.Environment(ws.entries) |
294 | + while requirements: |
295 | + # Process dependencies breadth-first. |
296 | + req = self._constrain(requirements.pop(0)) |
297 | + if req in processed: |
298 | + # Ignore cyclic or redundant dependencies. |
299 | + continue |
300 | + dist = best.get(req.key) |
301 | + if dist is None: |
302 | + # Find the best distribution and add it to the map. |
303 | + dist = ws.by_key.get(req.key) |
304 | + if dist is None: |
305 | + try: |
306 | + dist = best[req.key] = env.best_match(req, ws) |
307 | + except pkg_resources.VersionConflict, err: |
308 | + raise VersionConflict(err, ws) |
309 | + if dist is None: |
310 | + if destination: |
311 | + logger.debug('Getting required %r', str(req)) |
312 | + else: |
313 | + logger.debug('Adding required %r', str(req)) |
314 | + _log_requirement(ws, req) |
315 | + for dist in self._get_dist(req, |
316 | + ws, self._always_unzip): |
317 | + ws.add(dist) |
318 | + self._maybe_add_setuptools(ws, dist) |
319 | + if dist not in req: |
320 | + # Oops, the "best" so far conflicts with a dependency. |
321 | + raise VersionConflict( |
322 | + pkg_resources.VersionConflict(dist, req), ws) |
323 | + requirements.extend(dist.requires(req.extras)[::-1]) |
324 | + processed[req] = True |
325 | + if dist.location in self._site_packages: |
326 | + logger.debug('Egg from site-packages: %s', dist) |
327 | return ws |
328 | |
329 | def build(self, spec, build_ext): |
330 | @@ -836,6 +980,18 @@ |
331 | Installer._prefer_final = bool(setting) |
332 | return old |
333 | |
334 | +def include_site_packages(setting=None): |
335 | + old = Installer._include_site_packages |
336 | + if setting is not None: |
337 | + Installer._include_site_packages = bool(setting) |
338 | + return old |
339 | + |
340 | +def allowed_eggs_from_site_packages(setting=None): |
341 | + old = Installer._allowed_eggs_from_site_packages |
342 | + if setting is not None: |
343 | + Installer._allowed_eggs_from_site_packages = tuple(setting) |
344 | + return old |
345 | + |
346 | def use_dependency_links(setting=None): |
347 | old = Installer._use_dependency_links |
348 | if setting is not None: |
349 | @@ -858,19 +1014,27 @@ |
350 | links=(), index=None, |
351 | executable=sys.executable, always_unzip=None, |
352 | path=None, working_set=None, newest=True, versions=None, |
353 | - use_dependency_links=None, allow_hosts=('*',)): |
354 | + use_dependency_links=None, allow_hosts=('*',), |
355 | + include_site_packages=None, allowed_eggs_from_site_packages=None): |
356 | installer = Installer(dest, links, index, executable, always_unzip, path, |
357 | newest, versions, use_dependency_links, |
358 | - allow_hosts=allow_hosts) |
359 | + allow_hosts=allow_hosts, |
360 | + include_site_packages=include_site_packages, |
361 | + allowed_eggs_from_site_packages= |
362 | + allowed_eggs_from_site_packages) |
363 | return installer.install(specs, working_set) |
364 | |
365 | |
366 | def build(spec, dest, build_ext, |
367 | links=(), index=None, |
368 | executable=sys.executable, |
369 | - path=None, newest=True, versions=None, allow_hosts=('*',)): |
370 | + path=None, newest=True, versions=None, allow_hosts=('*',), |
371 | + include_site_packages=None, allowed_eggs_from_site_packages=None): |
372 | installer = Installer(dest, links, index, executable, True, path, newest, |
373 | - versions, allow_hosts=allow_hosts) |
374 | + versions, allow_hosts=allow_hosts, |
375 | + include_site_packages=include_site_packages, |
376 | + allowed_eggs_from_site_packages= |
377 | + allowed_eggs_from_site_packages) |
378 | return installer.build(spec, build_ext) |
379 | |
380 | |
381 | @@ -965,9 +1129,12 @@ |
382 | undo.reverse() |
383 | [f() for f in undo] |
384 | |
385 | - |
386 | -def working_set(specs, executable, path): |
387 | - return install(specs, None, executable=executable, path=path) |
388 | +def working_set(specs, executable, path, include_site_packages=None, |
389 | + allowed_eggs_from_site_packages=None): |
390 | + return install( |
391 | + specs, None, executable=executable, path=path, |
392 | + include_site_packages=include_site_packages, |
393 | + allowed_eggs_from_site_packages=allowed_eggs_from_site_packages) |
394 | |
395 | ############################################################################ |
396 | # Script generation functions |
397 | @@ -1002,7 +1169,7 @@ |
398 | def sitepackage_safe_scripts( |
399 | dest, working_set, executable, site_py_dest, |
400 | reqs=(), scripts=None, interpreter=None, extra_paths=(), |
401 | - initialization='', add_site_packages=False, exec_sitecustomize=False, |
402 | + initialization='', include_site_packages=False, exec_sitecustomize=False, |
403 | relative_paths=False, script_arguments='', script_initialization=''): |
404 | """Generate scripts and/or an interpreter from a system Python. |
405 | |
406 | @@ -1016,7 +1183,7 @@ |
407 | site_py_dest, executable, initialization, exec_sitecustomize)) |
408 | generated.append(_generate_site( |
409 | site_py_dest, working_set, executable, extra_paths, |
410 | - add_site_packages, relative_paths)) |
411 | + include_site_packages, relative_paths)) |
412 | script_initialization = ( |
413 | '\nimport site # imports custom buildout-generated site.py\n%s' % ( |
414 | script_initialization,)) |
415 | @@ -1301,54 +1468,6 @@ |
416 | |
417 | # These are used only by the newer ``sitepackage_safe_scripts`` function. |
418 | |
419 | -def _get_system_paths(executable): |
420 | - """Return lists of standard lib and site paths for executable. |
421 | - """ |
422 | - # We want to get a list of the site packages, which is not easy. |
423 | - # The canonical way to do this is to use |
424 | - # distutils.sysconfig.get_python_lib(), but that only returns a |
425 | - # single path, which does not reflect reality for many system |
426 | - # Pythons, which have multiple additions. Instead, we start Python |
427 | - # with -S, which does not import site.py and set up the extra paths |
428 | - # like site-packages or (Ubuntu/Debian) dist-packages and |
429 | - # python-support. We then compare that sys.path with the normal one |
430 | - # (minus user packages if this is Python 2.6, because we don't |
431 | - # support those (yet?). The set of the normal one minus the set of |
432 | - # the ones in ``python -S`` is the set of packages that are |
433 | - # effectively site-packages. |
434 | - # |
435 | - # The given executable might not be the current executable, so it is |
436 | - # appropriate to do another subprocess to figure out what the |
437 | - # additional site-package paths are. Moreover, even if this |
438 | - # executable *is* the current executable, this code might be run in |
439 | - # the context of code that has manipulated the sys.path--for |
440 | - # instance, to add local zc.buildout or setuptools eggs. |
441 | - def get_sys_path(*args, **kwargs): |
442 | - cmd = [executable] |
443 | - cmd.extend(args) |
444 | - cmd.extend([ |
445 | - "-c", "import sys, os;" |
446 | - "print repr([os.path.normpath(p) for p in sys.path if p])"]) |
447 | - # Windows needs some (as yet to be determined) part of the real env. |
448 | - env = os.environ.copy() |
449 | - env.update(kwargs) |
450 | - _proc = subprocess.Popen( |
451 | - cmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, env=env) |
452 | - stdout, stderr = _proc.communicate(); |
453 | - if _proc.returncode: |
454 | - raise RuntimeError( |
455 | - 'error trying to get system packages:\n%s' % (stderr,)) |
456 | - res = eval(stdout.strip()) |
457 | - try: |
458 | - res.remove('.') |
459 | - except ValueError: |
460 | - pass |
461 | - return res |
462 | - stdlib = get_sys_path('-S') # stdlib only |
463 | - no_user_paths = get_sys_path(PYTHONNOUSERSITE='x') |
464 | - site_paths = [p for p in no_user_paths if p not in stdlib] |
465 | - return (stdlib, site_paths) |
466 | - |
467 | def _get_module_file(executable, name): |
468 | """Return a module's file path. |
469 | |
470 | @@ -1401,10 +1520,10 @@ |
471 | return sitecustomize_path |
472 | |
473 | def _generate_site(dest, working_set, executable, extra_paths=(), |
474 | - add_site_packages=False, relative_paths=False): |
475 | + include_site_packages=False, relative_paths=False): |
476 | """Write a site.py file with eggs from working_set. |
477 | |
478 | - extra_paths will be added to the path. If add_site_packages is True, |
479 | + extra_paths will be added to the path. If include_site_packages is True, |
480 | paths from the underlying Python will be added. |
481 | """ |
482 | path = _get_path(working_set, extra_paths) |
483 | @@ -1416,7 +1535,7 @@ |
484 | [(line and ' %s' % (line,) or line) |
485 | for line in preamble.split('\n')]) |
486 | original_path_setup = '' |
487 | - if add_site_packages: |
488 | + if include_site_packages: |
489 | stdlib, site_paths = _get_system_paths(executable) |
490 | original_path_setup = original_path_snippet % ( |
491 | _format_paths((repr(p) for p in site_paths), 2),) |
492 | @@ -1431,7 +1550,7 @@ |
493 | relative_paths) |
494 | else: |
495 | location = repr(distribution.location) |
496 | - preamble += namespace_add_site_packages_setup % (location,) |
497 | + preamble += namespace_include_site_packages_setup % (location,) |
498 | original_path_setup = ( |
499 | addsitedir_namespace_originalpackages_snippet + |
500 | original_path_setup) |
501 | @@ -1460,7 +1579,7 @@ |
502 | raise RuntimeError('Buildout did not successfully rewrite site.py') |
503 | return site_path |
504 | |
505 | -namespace_add_site_packages_setup = ''' |
506 | +namespace_include_site_packages_setup = ''' |
507 | setuptools_path = %s |
508 | sys.path.append(setuptools_path) |
509 | known_paths.add(os.path.normcase(setuptools_path)) |
510 | |
511 | === modified file 'src/zc/buildout/easy_install.txt' |
512 | --- src/zc/buildout/easy_install.txt 2010-02-24 23:22:13 +0000 |
513 | +++ src/zc/buildout/easy_install.txt 2010-02-24 23:22:13 +0000 |
514 | @@ -89,6 +89,14 @@ |
515 | for using dependency_links in preference to other |
516 | locations. Defaults to true. |
517 | |
518 | +include_site_packages |
519 | + A flag indicating whether Python's non-standard-library packages should |
520 | + be available for finding dependencies. Defaults to true. |
521 | + |
522 | + Paths outside of Python's standard library--or more precisely, those that |
523 | + are not included when Python is started with the -S argument--are loosely |
524 | + referred to as "site-packages" here. |
525 | + |
526 | relative_paths |
527 | Adjust egg paths so they are relative to the script path. This |
528 | allows scripts to work when scripts and eggs are moved, as long as |
529 | @@ -399,6 +407,68 @@ |
530 | >>> [d.version for d in ws] |
531 | ['0.3', '1.1'] |
532 | |
533 | +Dependencies in Site Packages |
534 | +----------------------------- |
535 | + |
536 | +Paths outside of Python's standard library--or more precisely, those that are |
537 | +not included when Python is started with the -S argument--are loosely referred |
538 | +to as "site-packages" here. These site-packages are searched by default for |
539 | +distributions. This can be disabled, so that, for instance, a system Python |
540 | +can be used with buildout, cleaned of any packages installed by a user or |
541 | +system package manager. |
542 | + |
543 | +The default behavior can be controlled and introspected using |
544 | +zc.buildout.easy_install.include_site_packages. |
545 | + |
546 | + >>> zc.buildout.easy_install.include_site_packages() |
547 | + True |
548 | + |
549 | +Here's an example of using a Python executable that includes our dependencies. |
550 | + |
551 | +Our "py_path" will have the "demoneeded," and "demo" packages available. |
552 | + We'll simply be asking for "demoneeded" here, but without any external |
553 | + index or links. |
554 | + |
555 | + >>> from zc.buildout.tests import create_sample_sys_install |
556 | + >>> py_path, site_packages_path = make_py() |
557 | + >>> create_sample_sys_install(site_packages_path) |
558 | + |
559 | + >>> example_dest = tmpdir('site-packages-example-install') |
560 | + >>> workingset = zc.buildout.easy_install.install( |
561 | + ... ['demoneeded'], example_dest, links=[], executable=py_path, |
562 | + ... index=None) |
563 | + >>> [dist.project_name for dist in workingset] |
564 | + ['demoneeded'] |
565 | + |
566 | +That worked fine. Let's try again with site packages not allowed. We'll |
567 | +change the policy by changing the default. Notice that the function for |
568 | +changing the default value returns the previous value. |
569 | + |
570 | + >>> zc.buildout.easy_install.include_site_packages(False) |
571 | + True |
572 | + |
573 | + >>> zc.buildout.easy_install.include_site_packages() |
574 | + False |
575 | + |
576 | + >>> zc.buildout.easy_install.clear_index_cache() |
577 | + >>> rmdir(example_dest) |
578 | + >>> example_dest = tmpdir('site-packages-example-install') |
579 | + >>> workingset = zc.buildout.easy_install.install( |
580 | + ... ['demoneeded'], example_dest, links=[], executable=py_path, |
581 | + ... index=None) |
582 | + Traceback (most recent call last): |
583 | + ... |
584 | + MissingDistribution: Couldn't find a distribution for 'demoneeded'. |
585 | + >>> zc.buildout.easy_install.clear_index_cache() |
586 | + |
587 | +Now we'll reset the default. |
588 | + |
589 | + >>> zc.buildout.easy_install.include_site_packages(True) |
590 | + False |
591 | + |
592 | + >>> zc.buildout.easy_install.include_site_packages() |
593 | + True |
594 | + |
595 | Dependency links |
596 | ---------------- |
597 | |
598 | @@ -1197,7 +1267,7 @@ |
599 | >>> reset_interpreter() |
600 | >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( |
601 | ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, |
602 | - ... interpreter='py', add_site_packages=True) |
603 | + ... interpreter='py', include_site_packages=True) |
604 | >>> sys.stdout.write('#\n'); cat(site_path) |
605 | ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
606 | #... |
607 | @@ -1266,7 +1336,7 @@ |
608 | ... links=[link_server, namespace_eggs], index=link_server+'index/') |
609 | >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( |
610 | ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, |
611 | - ... interpreter='py', add_site_packages=True) |
612 | + ... interpreter='py', include_site_packages=True) |
613 | >>> sys.stdout.write('#\n'); cat(site_path) |
614 | ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
615 | #... |
616 | @@ -1324,7 +1394,7 @@ |
617 | >>> reset_interpreter() |
618 | >>> generated = zc.buildout.easy_install.sitepackage_safe_scripts( |
619 | ... interpreter_bin_dir, ws, sys.executable, interpreter_parts_dir, |
620 | - ... interpreter='py', add_site_packages=True, |
621 | + ... interpreter='py', include_site_packages=True, |
622 | ... relative_paths=interpreter_dir) |
623 | >>> sys.stdout.write('#\n'); cat(site_path) |
624 | ... # doctest: +ELLIPSIS +NORMALIZE_WHITESPACE |
625 | |
626 | === modified file 'src/zc/buildout/testing.py' |
627 | --- src/zc/buildout/testing.py 2010-02-24 23:22:13 +0000 |
628 | +++ src/zc/buildout/testing.py 2010-02-24 23:22:13 +0000 |
629 | @@ -222,11 +222,37 @@ |
630 | time.sleep(0.01) |
631 | raise ValueError('Timed out waiting for: '+label) |
632 | |
633 | +def get_installer_values(): |
634 | + """Get the current values for the easy_install module. |
635 | + |
636 | + This is necessary because instantiating a Buildout will force the |
637 | + Buildout's values on the installer. |
638 | + |
639 | + Returns a dict of names-values suitable for set_installer_values.""" |
640 | + names = ('default_versions', 'download_cache', 'install_from_cache', |
641 | + 'prefer_final', 'include_site_packages', |
642 | + 'allowed_eggs_from_site_packages', 'use_dependency_links', |
643 | + 'allow_picked_versions', 'always_unzip' |
644 | + ) |
645 | + values = {} |
646 | + for name in names: |
647 | + values[name] = getattr(zc.buildout.easy_install, name)() |
648 | + return values |
649 | + |
650 | +def set_installer_values(values): |
651 | + """Set the given values on the installer.""" |
652 | + for name, value in values.items(): |
653 | + getattr(zc.buildout.easy_install, name)(value) |
654 | + |
655 | def make_buildout(): |
656 | - # Create a basic buildout.cfg to avoid a warning from buildout: |
657 | + """Make a buildout that uses this version of zc.buildout.""" |
658 | + # Create a basic buildout.cfg to avoid a warning from buildout. |
659 | open('buildout.cfg', 'w').write( |
660 | "[buildout]\nparts =\n" |
661 | ) |
662 | + # Get state of installer defaults so we can reinstate them (instantiating |
663 | + # a Buildout will force the Buildout's defaults on the installer). |
664 | + installer_values = get_installer_values() |
665 | # Use the buildout bootstrap command to create a buildout |
666 | zc.buildout.buildout.Buildout( |
667 | 'buildout.cfg', |
668 | @@ -234,20 +260,23 @@ |
669 | # trick bootstrap into putting the buildout develop egg |
670 | # in the eggs dir. |
671 | ('buildout', 'develop-eggs-directory', 'eggs'), |
672 | - ] |
673 | + ], |
674 | + user_defaults=False, |
675 | ).bootstrap([]) |
676 | # Create the develop-eggs dir, which didn't get created the usual |
677 | # way due to the trick above: |
678 | os.mkdir('develop-eggs') |
679 | + # Reinstate the default values of the installer. |
680 | + set_installer_values(installer_values) |
681 | |
682 | def buildoutSetUp(test): |
683 | |
684 | test.globs['__tear_downs'] = __tear_downs = [] |
685 | test.globs['register_teardown'] = register_teardown = __tear_downs.append |
686 | |
687 | - prefer_final = zc.buildout.easy_install.prefer_final() |
688 | + installer_values = get_installer_values() |
689 | register_teardown( |
690 | - lambda: zc.buildout.easy_install.prefer_final(prefer_final) |
691 | + lambda: set_installer_values(installer_values) |
692 | ) |
693 | |
694 | here = os.getcwd() |
695 | @@ -367,8 +396,6 @@ |
696 | make_py = make_py |
697 | )) |
698 | |
699 | - zc.buildout.easy_install.prefer_final(prefer_final) |
700 | - |
701 | def buildoutTearDown(test): |
702 | for f in test.globs['__tear_downs']: |
703 | f() |
704 | |
705 | === modified file 'src/zc/buildout/tests.py' |
706 | --- src/zc/buildout/tests.py 2010-02-24 23:22:13 +0000 |
707 | +++ src/zc/buildout/tests.py 2010-02-24 23:22:13 +0000 |
708 | @@ -385,6 +385,64 @@ |
709 | Error: Couldn't find a distribution for 'demoneeded'. |
710 | """ |
711 | |
712 | +def show_eggs_from_site_packages(): |
713 | + """ |
714 | +Sometimes you want to know what eggs are coming from site-packages. This |
715 | +might be for a diagnostic, or so that you can get a starting value for the |
716 | +allowed-eggs-from-site-packages option. The -v flag will also include this |
717 | +information. |
718 | + |
719 | +Our "py_path" has the "demoneeded," "demo" |
720 | +packages available. We'll ask for "bigdemo," which will get both of them. |
721 | + |
722 | +Here's our set up. |
723 | + |
724 | + >>> py_path, site_packages_path = make_py() |
725 | + >>> create_sample_sys_install(site_packages_path) |
726 | + |
727 | + >>> write('buildout.cfg', |
728 | + ... ''' |
729 | + ... [buildout] |
730 | + ... parts = eggs |
731 | + ... prefer-final = true |
732 | + ... find-links = %(link_server)s |
733 | + ... |
734 | + ... [primed_python] |
735 | + ... executable = %(py_path)s |
736 | + ... |
737 | + ... [eggs] |
738 | + ... recipe = zc.recipe.egg:eggs |
739 | + ... python = primed_python |
740 | + ... eggs = bigdemo |
741 | + ... ''' % globals()) |
742 | + |
743 | +Now here is the output. The lines that begin with "Egg from site-packages:" |
744 | +indicate the eggs from site-packages that have been selected. You'll see |
745 | +we have two: demo 0.3 and demoneeded 1.1. |
746 | + |
747 | + >>> print system(py_path+" "+buildout+" -v") |
748 | + Installing 'zc.buildout', 'setuptools'. |
749 | + We have a develop egg: zc.buildout V |
750 | + We have the best distribution that satisfies 'setuptools'. |
751 | + Picked: setuptools = V |
752 | + Installing 'zc.recipe.egg'. |
753 | + We have a develop egg: zc.recipe.egg V |
754 | + Installing eggs. |
755 | + Installing 'bigdemo'. |
756 | + We have no distributions for bigdemo that satisfies 'bigdemo'. |
757 | + Getting distribution for 'bigdemo'. |
758 | + Got bigdemo 0.1. |
759 | + Picked: bigdemo = 0.1 |
760 | + Getting required 'demo' |
761 | + required by bigdemo 0.1. |
762 | + We have a develop egg: demo V |
763 | + Egg from site-packages: demo 0.3 |
764 | + Getting required 'demoneeded' |
765 | + required by demo 0.3. |
766 | + We have a develop egg: demoneeded V |
767 | + Egg from site-packages: demoneeded 1.1 |
768 | + <BLANKLINE> |
769 | + """ |
770 | |
771 | def test_comparing_saved_options_with_funny_characters(): |
772 | """ |
773 | @@ -1854,7 +1912,7 @@ |
774 | ... recipe = z3c.recipe.scripts |
775 | ... python = primed_python |
776 | ... interpreter = py |
777 | - ... add-site-packages = true |
778 | + ... include-site-packages = true |
779 | ... eggs = tellmy.version == 1.0 |
780 | ... tellmy.fortune == 1.0 |
781 | ... demo |
782 | @@ -1879,7 +1937,7 @@ |
783 | Generated interpreter '/sample-buildout/bin/py'. |
784 | <BLANKLINE> |
785 | |
786 | -Finally, we are ready for the actual test. Prior to the bug fix that |
787 | +Finally, we are ready to see if it worked. Prior to the bug fix that |
788 | this tests, the results of both calls below was the following:: |
789 | |
790 | 1.1 |
791 | @@ -2005,6 +2063,197 @@ |
792 | |
793 | """ |
794 | |
795 | +def isolated_include_site_packages(): |
796 | + """ |
797 | + |
798 | +This is an isolated test of the include_site_packages functionality, passing |
799 | +the argument directly to install, overriding a default. |
800 | + |
801 | +Our "py_path" has the "demoneeded" and "demo" packages available. We'll |
802 | +simply be asking for "demoneeded" here. |
803 | + |
804 | + >>> py_path, site_packages_path = make_py() |
805 | + >>> create_sample_sys_install(site_packages_path) |
806 | + >>> zc.buildout.easy_install.include_site_packages(False) |
807 | + True |
808 | + |
809 | + >>> example_dest = tmpdir('site-packages-example-install') |
810 | + >>> workingset = zc.buildout.easy_install.install( |
811 | + ... ['demoneeded'], example_dest, links=[], executable=py_path, |
812 | + ... index=None, include_site_packages=True) |
813 | + >>> [dist.project_name for dist in workingset] |
814 | + ['demoneeded'] |
815 | + |
816 | +That worked fine. Let's try again with site packages not allowed (and |
817 | +reversing the default). |
818 | + |
819 | + >>> zc.buildout.easy_install.include_site_packages(True) |
820 | + False |
821 | + |
822 | + >>> zc.buildout.easy_install.clear_index_cache() |
823 | + >>> rmdir(example_dest) |
824 | + >>> example_dest = tmpdir('site-packages-example-install') |
825 | + >>> workingset = zc.buildout.easy_install.install( |
826 | + ... ['demoneeded'], example_dest, links=[], executable=py_path, |
827 | + ... index=None, include_site_packages=False) |
828 | + Traceback (most recent call last): |
829 | + ... |
830 | + MissingDistribution: Couldn't find a distribution for 'demoneeded'. |
831 | + |
832 | +That's a failure, as expected. |
833 | + |
834 | +Now we explore an important edge case. |
835 | + |
836 | +Some system Pythons include setuptools (and other Python packages) in their |
837 | +site-packages (or equivalent) using a .egg-info directory. The pkg_resources |
838 | +module (from setuptools) considers a package installed using .egg-info to be a |
839 | +develop egg. |
840 | + |
841 | +zc.buildout.buildout.Buildout.bootstrap will make setuptools and zc.buildout |
842 | +available to the buildout via the eggs directory, for normal eggs; or the |
843 | +develop-eggs directory, for develop-eggs. |
844 | + |
845 | +If setuptools or zc.buildout is found in site-packages and considered by |
846 | +pkg_resources to be a develop egg, then the bootstrap code will use a .egg-link |
847 | +in the local develop-eggs, pointing to site-packages, in its entirety. Because |
848 | +develop-eggs must always be available for searching for distributions, this |
849 | +indirectly brings site-packages back into the search path for distributions. |
850 | + |
851 | +Because of this, we have to take special care that we still exclude |
852 | +site-packages even in this case. See the comments about site packages in the |
853 | +Installer._satisfied and Installer._obtain methods for the implementation |
854 | +(as of this writing). |
855 | + |
856 | +In this demonstration, we insert a link to the "demoneeded" distribution |
857 | +in our develop-eggs, which would bring the package back in, except for |
858 | +the special care we have taken to exclude it. |
859 | + |
860 | + >>> zc.buildout.easy_install.clear_index_cache() |
861 | + >>> rmdir(example_dest) |
862 | + >>> example_dest = tmpdir('site-packages-example-install') |
863 | + >>> mkdir(example_dest, 'develop-eggs') |
864 | + >>> write(example_dest, 'develop-eggs', 'demoneeded.egg-link', |
865 | + ... site_packages_path) |
866 | + >>> workingset = zc.buildout.easy_install.install( |
867 | + ... ['demoneeded'], example_dest, links=[], |
868 | + ... path=[join(example_dest, 'develop-eggs')], |
869 | + ... executable=py_path, |
870 | + ... index=None, include_site_packages=False) |
871 | + Traceback (most recent call last): |
872 | + ... |
873 | + MissingDistribution: Couldn't find a distribution for 'demoneeded'. |
874 | + |
875 | +The MissingDistribution error shows that buildout correctly excluded the |
876 | +"site-packages" source even though it was indirectly included in the path |
877 | +via a .egg-link file. |
878 | + |
879 | + """ |
880 | + |
881 | +def allowed_eggs_from_site_packages(): |
882 | + """ |
883 | +Sometimes you need or want to control what eggs from site-packages are used. |
884 | +The allowed-eggs-from-site-packages option allows you to specify a whitelist of |
885 | +project names that may be included from site-packages. You can use globs to |
886 | +specify the value. It defaults to a single value of '*', indicating that any |
887 | +package may come from site-packages. |
888 | + |
889 | +This option interacts with include-site-packages in the following ways. |
890 | + |
891 | +If include-site-packages is true, then allowed-eggs-from-site-packages filters |
892 | +what eggs from site-packages may be chosen. If allowed-eggs-from-site-packages |
893 | +is an empty list, then no eggs from site-packages are chosen, but site-packages |
894 | +will still be included at the end of path lists. |
895 | + |
896 | +If include-site-packages is false, allowed-eggs-from-site-packages is |
897 | +irrelevant. |
898 | + |
899 | +This test shows the interaction with the zc.buildout.easy_install API. Another |
900 | +test below (allow_site_package_eggs_option) shows using it with a buildout.cfg. |
901 | + |
902 | +Our "py_path" has the "demoneeded" and "demo" packages available. We'll |
903 | +simply be asking for "demoneeded" here. |
904 | + |
905 | + >>> py_path, site_packages_path = make_py() |
906 | + >>> create_sample_sys_install(site_packages_path) |
907 | + |
908 | + >>> example_dest = tmpdir('site-packages-example-install') |
909 | + >>> workingset = zc.buildout.easy_install.install( |
910 | + ... ['demoneeded'], example_dest, links=[], executable=py_path, |
911 | + ... index=None, |
912 | + ... allowed_eggs_from_site_packages=['demoneeded', 'other']) |
913 | + >>> [dist.project_name for dist in workingset] |
914 | + ['demoneeded'] |
915 | + |
916 | +That worked fine. It would work fine for a glob too. |
917 | + |
918 | + >>> zc.buildout.easy_install.clear_index_cache() |
919 | + >>> rmdir(example_dest) |
920 | + >>> example_dest = tmpdir('site-packages-example-install') |
921 | + >>> workingset = zc.buildout.easy_install.install( |
922 | + ... ['demoneeded'], example_dest, links=[], executable=py_path, |
923 | + ... index=None, |
924 | + ... allowed_eggs_from_site_packages=['?emon*', 'other']) |
925 | + >>> [dist.project_name for dist in workingset] |
926 | + ['demoneeded'] |
927 | + |
928 | +But now let's try again with 'demoneeded' not allowed. |
929 | + |
930 | + >>> zc.buildout.easy_install.clear_index_cache() |
931 | + >>> rmdir(example_dest) |
932 | + >>> example_dest = tmpdir('site-packages-example-install') |
933 | + >>> workingset = zc.buildout.easy_install.install( |
934 | + ... ['demoneeded'], example_dest, links=[], executable=py_path, |
935 | + ... index=None, |
936 | + ... allowed_eggs_from_site_packages=['demo']) |
937 | + Traceback (most recent call last): |
938 | + ... |
939 | + MissingDistribution: Couldn't find a distribution for 'demoneeded'. |
940 | + |
941 | +Here's the same, but with an empty list. |
942 | + |
943 | + >>> zc.buildout.easy_install.clear_index_cache() |
944 | + >>> rmdir(example_dest) |
945 | + >>> example_dest = tmpdir('site-packages-example-install') |
946 | + >>> workingset = zc.buildout.easy_install.install( |
947 | + ... ['demoneeded'], example_dest, links=[], executable=py_path, |
948 | + ... index=None, |
949 | + ... allowed_eggs_from_site_packages=[]) |
950 | + Traceback (most recent call last): |
951 | + ... |
952 | + MissingDistribution: Couldn't find a distribution for 'demoneeded'. |
953 | + |
954 | +Of course, this doesn't stop us from getting a package from elsewhere. Here, |
955 | +we add a link server. |
956 | + |
957 | + >>> zc.buildout.easy_install.clear_index_cache() |
958 | + >>> rmdir(example_dest) |
959 | + >>> example_dest = tmpdir('site-packages-example-install') |
960 | + >>> workingset = zc.buildout.easy_install.install( |
961 | + ... ['demoneeded'], example_dest, executable=py_path, |
962 | + ... links=[link_server], index=link_server+'index/', |
963 | + ... allowed_eggs_from_site_packages=['other']) |
964 | + >>> [dist.project_name for dist in workingset] |
965 | + ['demoneeded'] |
966 | + >>> [dist.location for dist in workingset] |
967 | + ['/site-packages-example-install/demoneeded-1.1-py2.6.egg'] |
968 | + |
969 | +Finally, here's an example of an interaction: we say that it is OK to |
970 | +allow the "demoneeded" egg to come from site-packages, but we don't |
971 | +include-site-packages. |
972 | + |
973 | + >>> zc.buildout.easy_install.clear_index_cache() |
974 | + >>> rmdir(example_dest) |
975 | + >>> example_dest = tmpdir('site-packages-example-install') |
976 | + >>> workingset = zc.buildout.easy_install.install( |
977 | + ... ['demoneeded'], example_dest, links=[], executable=py_path, |
978 | + ... index=None, include_site_packages=False, |
979 | + ... allowed_eggs_from_site_packages=['demoneeded']) |
980 | + Traceback (most recent call last): |
981 | + ... |
982 | + MissingDistribution: Couldn't find a distribution for 'demoneeded'. |
983 | + |
984 | + """ |
985 | + |
986 | if sys.version_info > (2, 4): |
987 | def test_exit_codes(): |
988 | """ |
989 | @@ -2932,24 +3181,59 @@ |
990 | finally: |
991 | shutil.rmtree(tmp) |
992 | |
993 | +def _write_eggrecipedemoneeded(tmp, minor_version, suffix=''): |
994 | + from zc.buildout.testing import write |
995 | + write(tmp, 'README.txt', '') |
996 | + write(tmp, 'eggrecipedemoneeded.py', |
997 | + 'y=%s\ndef f():\n pass' % minor_version) |
998 | + write( |
999 | + tmp, 'setup.py', |
1000 | + "from setuptools import setup\n" |
1001 | + "setup(name='demoneeded', py_modules=['eggrecipedemoneeded']," |
1002 | + " zip_safe=True, version='1.%s%s', author='bob', url='bob', " |
1003 | + "author_email='bob')\n" |
1004 | + % (minor_version, suffix) |
1005 | + ) |
1006 | + |
1007 | +def _write_eggrecipedemo(tmp, minor_version, suffix=''): |
1008 | + from zc.buildout.testing import write |
1009 | + write(tmp, 'README.txt', '') |
1010 | + write( |
1011 | + tmp, 'eggrecipedemo.py', |
1012 | + 'import eggrecipedemoneeded\n' |
1013 | + 'x=%s\n' |
1014 | + 'def main(): print x, eggrecipedemoneeded.y\n' |
1015 | + % minor_version) |
1016 | + write( |
1017 | + tmp, 'setup.py', |
1018 | + "from setuptools import setup\n" |
1019 | + "setup(name='demo', py_modules=['eggrecipedemo']," |
1020 | + " install_requires = 'demoneeded'," |
1021 | + " entry_points={'console_scripts': " |
1022 | + "['demo = eggrecipedemo:main']}," |
1023 | + " zip_safe=True, version='0.%s%s')\n" % (minor_version, suffix) |
1024 | + ) |
1025 | + |
1026 | +def create_sample_sys_install(site_packages_path): |
1027 | + for creator, minor_version in ( |
1028 | + (_write_eggrecipedemoneeded, 1), |
1029 | + (_write_eggrecipedemo, 3)): |
1030 | + # Write the files and install in site_packages_path. |
1031 | + tmp = tempfile.mkdtemp() |
1032 | + try: |
1033 | + creator(tmp, minor_version) |
1034 | + zc.buildout.testing.sys_install(tmp, site_packages_path) |
1035 | + finally: |
1036 | + shutil.rmtree(tmp) |
1037 | + |
1038 | def create_sample_eggs(test, executable=sys.executable): |
1039 | - write = test.globs['write'] |
1040 | + from zc.buildout.testing import write |
1041 | dest = test.globs['sample_eggs'] |
1042 | tmp = tempfile.mkdtemp() |
1043 | try: |
1044 | - write(tmp, 'README.txt', '') |
1045 | - |
1046 | for i in (0, 1, 2): |
1047 | - write(tmp, 'eggrecipedemoneeded.py', 'y=%s\ndef f():\n pass' % i) |
1048 | - c1 = i==2 and 'c1' or '' |
1049 | - write( |
1050 | - tmp, 'setup.py', |
1051 | - "from setuptools import setup\n" |
1052 | - "setup(name='demoneeded', py_modules=['eggrecipedemoneeded']," |
1053 | - " zip_safe=True, version='1.%s%s', author='bob', url='bob', " |
1054 | - "author_email='bob')\n" |
1055 | - % (i, c1) |
1056 | - ) |
1057 | + suffix = i==2 and 'c1' or '' |
1058 | + _write_eggrecipedemoneeded(tmp, i, suffix) |
1059 | zc.buildout.testing.sdist(tmp, dest) |
1060 | |
1061 | write( |
1062 | @@ -2963,22 +3247,8 @@ |
1063 | os.remove(os.path.join(tmp, 'eggrecipedemoneeded.py')) |
1064 | |
1065 | for i in (1, 2, 3, 4): |
1066 | - write( |
1067 | - tmp, 'eggrecipedemo.py', |
1068 | - 'import eggrecipedemoneeded\n' |
1069 | - 'x=%s\n' |
1070 | - 'def main(): print x, eggrecipedemoneeded.y\n' |
1071 | - % i) |
1072 | - c1 = i==4 and 'c1' or '' |
1073 | - write( |
1074 | - tmp, 'setup.py', |
1075 | - "from setuptools import setup\n" |
1076 | - "setup(name='demo', py_modules=['eggrecipedemo']," |
1077 | - " install_requires = 'demoneeded'," |
1078 | - " entry_points={'console_scripts': " |
1079 | - "['demo = eggrecipedemo:main']}," |
1080 | - " zip_safe=True, version='0.%s%s')\n" % (i, c1) |
1081 | - ) |
1082 | + suffix = i==4 and 'c1' or '' |
1083 | + _write_eggrecipedemo(tmp, i, suffix) |
1084 | zc.buildout.testing.bdist_egg(tmp, executable, dest) |
1085 | |
1086 | write(tmp, 'eggrecipebigdemo.py', 'import eggrecipedemo') |
1087 | |
1088 | === modified file 'z3c.recipe.scripts_/src/z3c/recipe/scripts/README.txt' |
1089 | --- z3c.recipe.scripts_/src/z3c/recipe/scripts/README.txt 2010-02-24 23:22:13 +0000 |
1090 | +++ z3c.recipe.scripts_/src/z3c/recipe/scripts/README.txt 2010-02-24 23:22:13 +0000 |
1091 | @@ -33,7 +33,7 @@ |
1092 | In addition to these, the recipe offers these new options. They are |
1093 | introduced here, and described more in depth below. |
1094 | |
1095 | -add-site-packages |
1096 | +include-site-packages |
1097 | You can choose to have the site-packages of the underlying Python |
1098 | available to your script or interpreter, in addition to the packages |
1099 | from your eggs. See the section on this option for motivations and |
1100 | @@ -47,7 +47,7 @@ |
1101 | exec-sitecustomize |
1102 | Normally the Python's real sitecustomize module is not processed. |
1103 | If you want it to be processed, set this value to 'true'. This will |
1104 | - be honored irrespective of the setting for add-site-packages. |
1105 | + be honored irrespective of the setting for include-site-packages. |
1106 | |
1107 | script-initialization |
1108 | The standard initialization code affects both an interpreter and scripts. |
1109 | @@ -202,7 +202,7 @@ |
1110 | possibilities. Don't be unaware of the dangers. |
1111 | |
1112 | To show off these features, we need to use buildout with a Python |
1113 | -executable with some extra paths to show ``add-site-packages``; and one |
1114 | +executable with some extra paths to show ``include-site-packages``; and one |
1115 | guaranteed to have a sitecustomize module to show |
1116 | ``exec-sitecustomize``. We'll make one using a test fixture called |
1117 | ``make_py``. The os.environ change below will go into the sitecustomize, |
1118 | @@ -215,7 +215,7 @@ |
1119 | >>> print site_packages_path |
1120 | /executable_buildout/site-packages |
1121 | |
1122 | -Now let's take a look at add-site-packages. |
1123 | +Now let's take a look at include-site-packages. |
1124 | |
1125 | >>> write(sample_buildout, 'buildout.cfg', |
1126 | ... """ |
1127 | @@ -225,7 +225,7 @@ |
1128 | ... |
1129 | ... [py] |
1130 | ... recipe = z3c.recipe.scripts:interpreter |
1131 | - ... add-site-packages = true |
1132 | + ... include-site-packages = true |
1133 | ... eggs = demo<0.3 |
1134 | ... find-links = %(server)s |
1135 | ... index = %(server)s/index |
1136 | |
1137 | === modified file 'z3c.recipe.scripts_/src/z3c/recipe/scripts/scripts.py' |
1138 | --- z3c.recipe.scripts_/src/z3c/recipe/scripts/scripts.py 2010-02-24 23:22:13 +0000 |
1139 | +++ z3c.recipe.scripts_/src/z3c/recipe/scripts/scripts.py 2010-02-24 23:22:13 +0000 |
1140 | @@ -31,13 +31,13 @@ |
1141 | b_options['parts-directory'], self.name) |
1142 | |
1143 | value = options.setdefault( |
1144 | - 'add-site-packages', |
1145 | - b_options.get('add-site-packages', 'false')) |
1146 | + 'include-site-packages', |
1147 | + b_options.get('include-site-packages', 'false')) |
1148 | if value not in ('true', 'false'): |
1149 | raise zc.buildout.UserError( |
1150 | - "Invalid value for add-site-packages option: %s" % |
1151 | + "Invalid value for include-site-packages option: %s" % |
1152 | (value,)) |
1153 | - self.add_site_packages = (value == 'true') |
1154 | + self.include_site_packages = (value == 'true') |
1155 | |
1156 | value = options.setdefault( |
1157 | 'exec-sitecustomize', |
1158 | @@ -69,7 +69,7 @@ |
1159 | interpreter=options['name'], |
1160 | extra_paths=self.extra_paths, |
1161 | initialization=options.get('initialization', ''), |
1162 | - add_site_packages=self.add_site_packages, |
1163 | + include_site_packages=self.include_site_packages, |
1164 | exec_sitecustomize=self.exec_sitecustomize, |
1165 | relative_paths=self._relative_paths, |
1166 | )) |
1167 | @@ -92,7 +92,7 @@ |
1168 | interpreter=options.get('interpreter'), |
1169 | extra_paths=self.extra_paths, |
1170 | initialization=options.get('initialization', ''), |
1171 | - add_site_packages=self.add_site_packages, |
1172 | + include_site_packages=self.include_site_packages, |
1173 | exec_sitecustomize=self.exec_sitecustomize, |
1174 | relative_paths=self._relative_paths, |
1175 | script_arguments=options.get('arguments', ''), |
1176 | |
1177 | === modified file 'z3c.recipe.scripts_/src/z3c/recipe/scripts/tests.py' |
1178 | --- z3c.recipe.scripts_/src/z3c/recipe/scripts/tests.py 2010-02-24 23:22:13 +0000 |
1179 | +++ z3c.recipe.scripts_/src/z3c/recipe/scripts/tests.py 2010-02-24 23:22:13 +0000 |
1180 | @@ -25,7 +25,7 @@ |
1181 | # all of the examples. The README tests ``extends``, |
1182 | # ``include-site-customization`` and ``name``. That leaves ``python``, |
1183 | # ``extra-paths``, ``initialization``, ``relative-paths``, and |
1184 | -# ``add-site-packages``. |
1185 | +# ``include-site-packages``. |
1186 | |
1187 | def supports_python_option(): |
1188 | """ |
This branch builds the infrastructure in easy_install that is needed for z3c.recipe.scripts to control what packages from site-packages can be used to fulfill setup.py dependencies. Hooking it up comes in the subsequent branch, ~gary/zc. buildout/ python- support- 6-egg-control .
This branch merged the remaining work from my previous zc.buildout effort. Much of this branch has therefore been reviewed before.
In the previous branch, I had a flag called "include- site-packages" (or "include_ site_packages" depending on whether it is a config file or Python) for behavior related to what I had been calling "add-site-packages" (or "add_site_ packages" ) in the more recent effort. I settled on "include- site-packages" /"include_ site_packages" because it seemed to me to better describe the behavior of the flag, particularly in the case of the zc.buildout. easy_install. include_ site_packages function.
My addition and use of _get_version_info is new to this branch. It seems OK to me, but I questioned it, and would welcome other ideas. You'll notice that previously buildout indiscriminately added the buildout and setuptools paths. I felt that being more careful was warranted, but maybe I'm wrong.
I noticed a couple of intermittent test failures during the development of this branch. I got them to go away with the use of get_installer_ values and set_installer_ values for careful teardowns of various test changes. This work is also new.
That's it. Thank you.
Gary