Merge lp:~barry/python/pep3147 into lp:python/py3k

Proposed by Barry Warsaw
Status: Merged
Merge reported by: Barry Warsaw
Merged at revision: not available
Proposed branch: lp:~barry/python/pep3147
Merge into: lp:python/py3k
Diff against target: 2808 lines (+1209/-293)
39 files modified
.bzrignore (+1/-0)
.hgignore (+1/-0)
Doc/c-api/import.rst (+17/-0)
Doc/library/compileall.rst (+12/-6)
Doc/library/imp.rst (+35/-2)
Doc/library/py_compile.rst (+7/-3)
Doc/library/runpy.rst (+4/-1)
Include/import.h (+3/-0)
Lib/compileall.py (+48/-26)
Lib/importlib/_bootstrap.py (+20/-1)
Lib/importlib/test/__main__.py (+6/-1)
Lib/importlib/test/source/test_file_loader.py (+5/-4)
Lib/importlib/test/source/test_finder.py (+12/-1)
Lib/importlib/test/source/test_source_encoding.py (+1/-1)
Lib/importlib/test/source/util.py (+11/-8)
Lib/importlib/util.py (+1/-0)
Lib/inspect.py (+1/-0)
Lib/py_compile.py (+27/-21)
Lib/pydoc.py (+2/-1)
Lib/runpy.py (+2/-0)
Lib/site.py (+8/-4)
Lib/test/script_helper.py (+12/-11)
Lib/test/support.py (+64/-26)
Lib/test/test_cmd_line_script.py (+18/-12)
Lib/test/test_compileall.py (+69/-10)
Lib/test/test_frozen.py (+5/-5)
Lib/test/test_imp.py (+125/-2)
Lib/test/test_import.py (+176/-30)
Lib/test/test_pkg.py (+10/-10)
Lib/test/test_pkgimport.py (+14/-12)
Lib/test/test_pydoc.py (+7/-9)
Lib/test/test_runpy.py (+17/-8)
Lib/test/test_site.py (+32/-13)
Lib/test/test_zipfile.py (+8/-1)
Lib/test/test_zipimport.py (+19/-20)
Lib/zipfile.py (+40/-13)
Makefile.pre.in (+1/-0)
Python/import.c (+366/-31)
Python/pythonrun.c (+2/-0)
To merge this branch: bzr merge lp:~barry/python/pep3147
Reviewer Review Type Date Requested Status
Python Development Pending
Review via email: mp+22648@code.launchpad.net

Description of the change

Work-in-progress implementation of PEP 3147

http://www.python.org/dev/peps/pep-3147/

To post a comment you must log in.
lp:~barry/python/pep3147 updated
41150. By raymond.hettinger

Document sorting HowTo

41151. By raymond.hettinger

Add and update itertools recipes.

41152. By raymond.hettinger

Fix nits in itertools recipes.

41153. By georg.brandl

#8213: document behavior of -u on py3k better.

41154. By larry.hastings

Blocked revisions 79590 via svnmerge

........
  r79590 | larry.hastings | 2010-04-02 04:01:35 -0700 (Fri, 02 Apr 2010) | 10 lines

  Capsule-related changes:
  * PyCObject_AsVoidPtr() can now open capsules. This addresses
    most of the remaining backwards-compatibility concerns about
    the conversion of Python 2.7 from CObjects to capsules.
  * CObjects were marked Pending Deprecation.
  * Documentation about this pending deprecation was added to
    cobject.h.
  * The capsule source files were added to the legacy PC build
    processes.
........

41155. By larry.hastings

Blocked revisions 79592 via svnmerge

........
  r79592 | larry.hastings | 2010-04-02 04:18:17 -0700 (Fri, 02 Apr 2010) | 3 lines

  Issue #8235: _socket: Add the constant ``SO_SETFIB``. SO_SETFIB is
  a socket option available on FreeBSD 7.1 and newer.
........

41156. By larry.hastings

- Issue #8235: _socket: Add the constant ``SO_SETFIB``. SO_SETFIB is
  a socket option available on FreeBSD 7.1 and newer.

41157. By ezio.melotti

Merged revisions 79540-79541 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

Add -Wd to the flags used to run the tests (-3 is not necessary here).

........
  r79540 | ezio.melotti | 2010-04-01 01:18:09 +0300 (Thu, 01 Apr 2010) | 1 line

  Add -Wd and -3 to the flags used to run the tests.
........
  r79541 | ezio.melotti | 2010-04-01 01:43:31 +0300 (Thu, 01 Apr 2010) | 1 line

  Add -Wd and -3 to the flags used to run the tests on Windows.
........

41158. By raymond.hettinger

Issue 8257: Decimal constructor to accept float argument.

41159. By andrew.kuchling

Merged revisions 79603 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79603 | andrew.kuchling | 2010-04-02 12:59:16 -0400 (Fri, 02 Apr 2010) | 1 line

  #4440: modernize a use of filter(), making it compatible with 3.x
........

41160. By antoine.pitrou

Furniture is not very reliable these days (buildbot failures).

41161. By florent.xicluna

(partially)

Merged revisions 79534,79537,79539,79558,79606 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79534 | florent.xicluna | 2010-03-31 23:21:54 +0200 (mer, 31 mar 2010) | 2 lines

  Fix test for xml.etree when using a non-ascii path. And use check_warnings instead of catch_warnings.
........
  r79537 | florent.xicluna | 2010-03-31 23:40:32 +0200 (mer, 31 mar 2010) | 2 lines

  Fix typo
........
  r79539 | florent.xicluna | 2010-04-01 00:01:03 +0200 (jeu, 01 avr 2010) | 2 lines

  Replace catch_warnings with check_warnings when it makes sense. Use assertRaises context manager to simplify some tests.
........
  r79558 | florent.xicluna | 2010-04-01 20:17:09 +0200 (jeu, 01 avr 2010) | 2 lines

  #7092: Fix some -3 warnings, and fix Lib/platform.py when the path contains a double-quote.
........
  r79606 | florent.xicluna | 2010-04-02 19:26:42 +0200 (ven, 02 avr 2010) | 2 lines

  Backport some robotparser test and skip the test if the external resource is not available.
........

41162. By raymond.hettinger

Add nice docstrings to namedtuples.

41163. By florent.xicluna

An oversight in previous merge, r79610: test_doctest is sensible to the line numbering.

41164. By georg.brandl

#8288: fix next -> __next__.

41165. By tarek.ziade

Merged revisions 79618 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79618 | tarek.ziade | 2010-04-02 23:14:04 +0200 (Fri, 02 Apr 2010) | 1 line

  removed the local copy of xxmodule, and skip only test_build_ext when xxmodule is not found, not the whole unittest
........

41166. By ezio.melotti

Blocked revisions 79582 via svnmerge

........
  r79582 | georg.brandl | 2010-04-02 11:51:31 +0300 (Fri, 02 Apr 2010) | 1 line

  Fix typo in unicode character name.
........

41167. By brian.curtin

Change test_support to support. Fixes a failing test on Windows.

41168. By benjamin.peterson

Blocked revisions 79636,79639 via svnmerge

........
  r79636 | benjamin.peterson | 2010-04-02 18:59:41 -0500 (Fri, 02 Apr 2010) | 4 lines

  always check _PyString_Resize for error

  also normalize how this error is checked
........
  r79639 | benjamin.peterson | 2010-04-02 19:57:33 -0500 (Fri, 02 Apr 2010) | 1 line

  more _PyString_Resize error checking
........

41169. By benjamin.peterson

Merged revisions 79642,79644 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79642 | benjamin.peterson | 2010-04-02 20:08:34 -0500 (Fri, 02 Apr 2010) | 1 line

  split out large test function
........
  r79644 | benjamin.peterson | 2010-04-02 20:28:57 -0500 (Fri, 02 Apr 2010) | 1 line

  give TypeError when trying to set T_STRING_INPLACE
........

41170. By victor.stinner

Blocked revisions 79654 via svnmerge

........
  r79654 | victor.stinner | 2010-04-03 10:40:16 +0200 (sam., 03 avril 2010) | 5 lines

  Issue #8227: Fix C API documentation, argument parsing

   * 'z', 'z#', 'z*' does also accept Unicode
   * unify types name: replace "string or Unicode objet" by "string or Unicode"
........

41171. By tarek.ziade

Merged revisions 79652 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79652 | tarek.ziade | 2010-04-03 10:37:59 +0200 (Sat, 03 Apr 2010) | 1 line

  Issue #1222585: Added LDCXXSHARED for C++ support
........

41172. By raymond.hettinger

Add a subtract() method to collections.Counter()

41173. By mark.dickinson

Merged revisions 79661 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79661 | mark.dickinson | 2010-04-03 11:27:05 +0100 (Sat, 03 Apr 2010) | 14 lines

  Fix a couple of issues with the test_structmembersType class in _testcapimodule

   - rename to _test_structmembersType to avoid the class being automatically
     called by test_capi

   - allow space for trailing NUL in inplace_member field of all_structmembers

   - use T_STRING_INPLACE instead of T_INPLACE_STRING as keyword argument
     to _test_structmembersType initializer

   - don't attempt to initialize inplace_member field if T_STRING_INPLACE
     argument wasn't supplied.
........

41174. By mark.dickinson

Merged revisions 79583,79588-79589 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79583 | mark.dickinson | 2010-04-02 09:53:22 +0100 (Fri, 02 Apr 2010) | 7 lines

  Issue #2531: Make float-to-decimal comparisons return correct results.

  Float to decimal comparison operations now return a result based on
  the numeric values of the operands. Decimal.__hash__ has also been
  fixed so that Decimal and float values that compare equal have equal
  hash value.
........
  r79588 | mark.dickinson | 2010-04-02 11:17:07 +0100 (Fri, 02 Apr 2010) | 2 lines

  Issue #7279: Make comparisons involving a Decimal sNaN signal InvalidOperation.
........
  r79589 | mark.dickinson | 2010-04-02 11:35:12 +0100 (Fri, 02 Apr 2010) | 6 lines

  Issue #7279: Make Decimal('nan') hashable. Decimal('snan') remains unhashable.

  Also rewrite the Decimal __hash__ method so that it doesn't rely on
  float('inf') being valid: float('inf') could raise an exception on
  platforms not using IEEE 754 arithmetic.
........

41175. By mark.dickinson

Merged revisions 79629 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79629 | mark.dickinson | 2010-04-02 23:27:36 +0100 (Fri, 02 Apr 2010) | 2 lines

  Issue #8294: Allow float and Decimal arguments in Fraction constructor.
........

41176. By mark.dickinson

Remove unused define.

41177. By ezio.melotti

Merged revisions 79678 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79678 | ezio.melotti | 2010-04-03 17:51:00 +0300 (Sat, 03 Apr 2010) | 1 line

  Use more specific assert* methods in test_decimal.
........

41178. By mark.dickinson

Clean up integer tests in test_struct, and bring them more in line with the trunk tests.

41179. By mark.dickinson

Internal refactoring in struct.pack: make all integer conversions go through get_pylong.

41180. By ezio.melotti

Add -bb to rt.bat to do some extra checks during the tests (it is already used by the non-Windows buildbots).

41181. By ezio.melotti

Blocked revisions 79624 via svnmerge

........
  r79624 | ezio.melotti | 2010-04-03 00:43:10 +0300 (Sat, 03 Apr 2010) | 1 line

  Fix test_compiler.py that was using unittest.__file__ to find Lib/ (unittest is now a package).
........

41182. By benjamin.peterson

Blocked revisions 79693,79695 via svnmerge

........
  r79693 | benjamin.peterson | 2010-04-03 10:38:38 -0500 (Sat, 03 Apr 2010) | 1 line

  wrap
........
  r79695 | benjamin.peterson | 2010-04-03 10:40:29 -0500 (Sat, 03 Apr 2010) | 1 line

  remove unneeded argument
........

41183. By mark.dickinson

Merged revisions 79699 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79699 | mark.dickinson | 2010-04-03 16:47:46 +0100 (Sat, 03 Apr 2010) | 1 line

  Add Misc/NEWS entry for r79609.
........

41184. By mark.dickinson

Merged revisions 79674 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79674 | mark.dickinson | 2010-04-03 15:05:10 +0100 (Sat, 03 Apr 2010) | 3 lines

  Issue #8300: Let struct.pack use __index__ to convert and pack non-integers.
  Based on a patch by Meador Inge.
........

41185. By benjamin.peterson

Blocked revisions 79697-79698,79704 via svnmerge

........
  r79697 | benjamin.peterson | 2010-04-03 10:44:56 -0500 (Sat, 03 Apr 2010) | 1 line

  silence PyCObject warnings in bsddb
........
  r79698 | benjamin.peterson | 2010-04-03 10:45:59 -0500 (Sat, 03 Apr 2010) | 1 line

  spelling
........
  r79704 | benjamin.peterson | 2010-04-03 10:58:15 -0500 (Sat, 03 Apr 2010) | 1 line

  remove deprecation warnings silence attempting
........

41186. By mark.dickinson

Blocked revisions 79707 via svnmerge

........
  r79707 | mark.dickinson | 2010-04-03 17:41:20 +0100 (Sat, 03 Apr 2010) | 5 lines

  Ensure 'module removed' warning messages contain the word 'module' or 'package'.

  This should fix the test_py3kwarn failure on OS X. test_support.import_module
  also requires this.
........

41187. By mark.dickinson

Blocked revisions 79714 via svnmerge

........
  r79714 | mark.dickinson | 2010-04-03 19:17:54 +0100 (Sat, 03 Apr 2010) | 1 line

  Silence DeprecationWarnings from uses of has_key and <> in plat-mac.
........

41188. By benjamin.peterson

Blocked revisions 79712 via svnmerge

........
  r79712 | raymond.hettinger | 2010-04-03 12:10:05 -0500 (Sat, 03 Apr 2010) | 1 line

  Silence a compiler warning.
........

41189. By benjamin.peterson

Blocked revisions 79718-79719 via svnmerge

........
  r79718 | antoine.pitrou | 2010-04-03 16:42:18 -0500 (Sat, 03 Apr 2010) | 3 lines

  Remove useless (?) import from r79706
........
  r79719 | benjamin.peterson | 2010-04-03 16:50:40 -0500 (Sat, 03 Apr 2010) | 1 line

  import bsddb more robustly
........

41190. By benjamin.peterson

Merged revisions 79723 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79723 | benjamin.peterson | 2010-04-03 17:48:51 -0500 (Sat, 03 Apr 2010) | 1 line

  ensure that the locale does not affect the tokenization of identifiers
........

41191. By benjamin.peterson

Merged revisions 79725 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79725 | benjamin.peterson | 2010-04-03 18:03:35 -0500 (Sat, 03 Apr 2010) | 4 lines

  use our own locale independent ctype macros

  requires building pyctype.o into pgen
........

41192. By raymond.hettinger

Add count() method to collections.deque().

41193. By benjamin.peterson

add missing decrefs

41194. By ezio.melotti

Fix versionchanged from 2.7 to 3.2 in decimal.rst.

41195. By ezio.melotti

Merged revisions 79733,79735 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79733 | ezio.melotti | 2010-04-04 09:41:27 +0300 (Sun, 04 Apr 2010) | 1 line

  Add a versionadded to Counter.subtract() doc.
........
  r79735 | ezio.melotti | 2010-04-04 09:50:58 +0300 (Sun, 04 Apr 2010) | 1 line

  Revert 79733 - Counter is new in 2.7, the versionadded is not necessary - and fix a typo.
........

41196. By ezio.melotti

Recorded merge of revisions 79660 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79660 | raymond.hettinger | 2010-04-03 13:22:00 +0300 (Sat, 03 Apr 2010) | 1 line

  Add subtract() method to collections.Counter() objects.
........

41197. By ezio.melotti

Merged revisions 79740 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79740 | ezio.melotti | 2010-04-04 10:00:02 +0300 (Sun, 04 Apr 2010) | 1 line

  Use more specific assert* methods in test_struct.
........

41198. By mark.dickinson

Merged revisions 79745 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79745 | mark.dickinson | 2010-04-04 09:43:04 +0100 (Sun, 04 Apr 2010) | 3 lines

  Issue #8300 (__index__ handling in struct.pack): Remove redundant check
  and improve test coverage. Thanks Meador Inge for the patch.
........

41199. By mark.dickinson

Merged revisions 79751 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79751 | mark.dickinson | 2010-04-04 22:19:35 +0100 (Sun, 04 Apr 2010) | 1 line

  A handful of whitespace fixes in Modules/_struct.c.
........

41200. By mark.dickinson

Merged revisions 79756 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79756 | mark.dickinson | 2010-04-04 23:09:21 +0100 (Sun, 04 Apr 2010) | 6 lines

  Add versionchanged entry for Decimal(float) construction.

  Also add an example of constructing a Decimal directly from a float,
  update the quickstart tutorial, and tweak another couple of
  sentences.
........

41201. By benjamin.peterson

Merged revisions 79762 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79762 | benjamin.peterson | 2010-04-04 18:01:03 -0500 (Sun, 04 Apr 2010) | 1 line

  tabify
........

41202. By benjamin.peterson

Merged revisions 79763 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79763 | benjamin.peterson | 2010-04-04 18:03:22 -0500 (Sun, 04 Apr 2010) | 1 line

  fix tabs
........

41203. By benjamin.peterson

Blocked revisions 79769 via svnmerge

........
  r79769 | benjamin.peterson | 2010-04-04 18:23:22 -0500 (Sun, 04 Apr 2010) | 1 line

  fix dis on new style classes #8310
........

41204. By benjamin.peterson

factor out constant

41205. By ezio.melotti

Merged revisions 79768 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79768 | ezio.melotti | 2010-04-05 02:21:53 +0300 (Mon, 05 Apr 2010) | 1 line

  Fix markup in decimal.rst.
........

41206. By philip.jenvey

Merged revisions 79779 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79779 | philip.jenvey | 2010-04-04 19:51:51 -0700 (Sun, 04 Apr 2010) | 2 lines

  fix escape_encode to return the correct consumed size
........

41207. By ezio.melotti

Merged revisions 79781 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79781 | ezio.melotti | 2010-04-05 06:51:38 +0300 (Mon, 05 Apr 2010) | 1 line

  #8212: rephrase the webservers howto and fix some mistakes.
........

41208. By ezio.melotti

Merged revisions 79788 via svnmerge from
svn+ssh://<email address hidden>/python/trunk

........
  r79788 | ezio.melotti | 2010-04-05 11:16:41 +0300 (Mon, 05 Apr 2010) | 1 line

  Fix markup errors.
........

41209. By Barry Warsaw

branch merge

41210. By Barry Warsaw

Ensure that no .pyc file outside of __pycache__ is written by the test suite.

41211. By Barry Warsaw

merge trunk

41212. By Barry Warsaw

Many updates based on Rietveld review by Antoine Pitrou.

41213. By Barry Warsaw

trunk merge

41214. By Barry Warsaw

trunk merge

41215. By Barry Warsaw

trunk merge

41216. By Barry Warsaw

Second round of Rietveld comments.

* Added documentation for cache_from_source(), source_from_cache(), and
  magic_tags.
* Use addCleanup() where appropriate.
* Other style cleanup.

41217. By Barry Warsaw

Test repairs.

41218. By Barry Warsaw

trunk merge

41219. By Barry Warsaw

Fixes for Windows compatibility.

41220. By Barry Warsaw

Be more aggressive for Windows.

41221. By Barry Warsaw

trunk merge

41222. By Barry Warsaw

Benjamin makes a good point that debug_override should accept true-ish and
false-ish values (when not None) to more closely mirror a pure-Python
implementation.

41223. By Barry Warsaw

* Add __cached__ attribute to modules.
* PyImport_ExecCodeModuleEx() gets a new parameter.

41224. By Barry Warsaw

Remove some unnecessary code.

41225. By Barry Warsaw

Based on Guido's recommendation, add PyImport_ExecCodeModuleExEx()

41226. By Barry Warsaw

trunk merge

41227. By Barry Warsaw

* PyImport_ExecCodeModuleExEx() -> PyImport_ExecCodeModuleWithPathnames()
* Document the above.
* Added PyImport_GetMagicTag() and document.
* Removed imp.magic_tags and replaced with imp.get_tag().

41228. By Barry Warsaw

More responses to Rietveld and mailing list comments. The most important
change here is an additional guard against buffer overflows in
make_compiled_pathname().

41229. By Barry Warsaw

trunk merge

41230. By Barry Warsaw

Patches from Brian Curtin for Windows.

41231. By Barry Warsaw

Support -b flag for compile to create legacy .pyc files. This does not delete
the source files.

41232. By Barry Warsaw

Document extended compileall options.

41233. By Barry Warsaw

trunk merge

41234. By Barry Warsaw

Make sure __cache__ shows up in modules executed via command line and -m
(runpy). Update runpy documentation.

41235. By Barry Warsaw

Typo.

41236. By Barry Warsaw

trunk merge

41237. By Barry Warsaw

* abs__file__() -> abs_paths() since this now also modifies m.__cached__
* actually make test_abs_path() not suck.

41238. By Barry Warsaw

trunk merge

41239. By Barry Warsaw

trunk merge

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file '.bzrignore'
--- .bzrignore 2010-04-11 20:49:28 +0000
+++ .bzrignore 2010-04-16 18:55:41 +0000
@@ -33,3 +33,4 @@
33Lib/test/data/*33Lib/test/data/*
34Lib/lib2to3/Grammar*.pickle34Lib/lib2to3/Grammar*.pickle
35Lib/lib2to3/PatternGrammar*.pickle35Lib/lib2to3/PatternGrammar*.pickle
36__pycache__
3637
=== modified file '.hgignore'
--- .hgignore 2010-04-13 16:37:01 +0000
+++ .hgignore 2010-04-16 18:55:41 +0000
@@ -54,3 +54,4 @@
54PCbuild/*.ncb54PCbuild/*.ncb
55PCbuild/*.bsc55PCbuild/*.bsc
56PCbuild/Win32-temp-*56PCbuild/Win32-temp-*
57__pycache__
5758
=== modified file 'Doc/c-api/import.rst'
--- Doc/c-api/import.rst 2010-04-11 15:12:57 +0000
+++ Doc/c-api/import.rst 2010-04-16 18:55:41 +0000
@@ -124,12 +124,24 @@
124 If *name* points to a dotted name of the form ``package.module``, any package124 If *name* points to a dotted name of the form ``package.module``, any package
125 structures not already created will still not be created.125 structures not already created will still not be created.
126126
127 See also :func:`PyImport_ExecCodeModuleEx` and
128 :func:`PyImport_ExecCodeModuleWithPathnames`.
129
127130
128.. cfunction:: PyObject* PyImport_ExecCodeModuleEx(char *name, PyObject *co, char *pathname)131.. cfunction:: PyObject* PyImport_ExecCodeModuleEx(char *name, PyObject *co, char *pathname)
129132
130 Like :cfunc:`PyImport_ExecCodeModule`, but the :attr:`__file__` attribute of133 Like :cfunc:`PyImport_ExecCodeModule`, but the :attr:`__file__` attribute of
131 the module object is set to *pathname* if it is non-``NULL``.134 the module object is set to *pathname* if it is non-``NULL``.
132135
136 See also :func:`PyImport_ExecCodeModuleWithPathnames`.
137
138
139.. cfunction:: PyObject* PyImport_ExecCodeModuleWithPathnames(char *name, PyObject *co, char *pathname, char *cpathname)
140
141 Like :cfunc:`PyImport_ExecCodeModuleEx`, but the :attr:`__cached__`
142 attribute of the module object is set to *cpathname* if it is
143 non-``NULL``. Of the three functions, this is the preferred one to use.
144
133145
134.. cfunction:: long PyImport_GetMagicNumber()146.. cfunction:: long PyImport_GetMagicNumber()
135147
@@ -138,6 +150,11 @@
138 of the bytecode file, in little-endian byte order.150 of the bytecode file, in little-endian byte order.
139151
140152
153.. cfunction:: const char * PyImport_GetMagicTag()
154
155 Return the magic tag string for :pep:`3147` format Python bytecode file
156 names.
157
141.. cfunction:: PyObject* PyImport_GetModuleDict()158.. cfunction:: PyObject* PyImport_GetModuleDict()
142159
143 Return the dictionary used for the module administration (a.k.a.160 Return the dictionary used for the module administration (a.k.a.
144161
=== modified file 'Doc/library/compileall.rst'
--- Doc/library/compileall.rst 2009-05-17 11:29:12 +0000
+++ Doc/library/compileall.rst 2010-04-16 18:55:41 +0000
@@ -17,9 +17,11 @@
17sys.path``. Printing lists of the files compiled can be disabled with the17sys.path``. Printing lists of the files compiled can be disabled with the
18:option:`-q` flag. In addition, the :option:`-x` option takes a regular18:option:`-q` flag. In addition, the :option:`-x` option takes a regular
19expression argument. All files that match the expression will be skipped.19expression argument. All files that match the expression will be skipped.
2020The :option:`-b` flag may be given to write legacy ``.pyc`` file path names,
2121otherwise :pep:`3147` style byte-compiled path names are written.
22.. function:: compile_dir(dir, maxlevels=10, ddir=None, force=False, rx=None, quiet=False)22
23
24.. function:: compile_dir(dir, maxlevels=10, ddir=None, force=False, rx=None, quiet=False, legacy=False)
2325
24 Recursively descend the directory tree named by *dir*, compiling all :file:`.py`26 Recursively descend the directory tree named by *dir*, compiling all :file:`.py`
25 files along the way. The *maxlevels* parameter is used to limit the depth of27 files along the way. The *maxlevels* parameter is used to limit the depth of
@@ -34,12 +36,16 @@
34 If *quiet* is true, nothing is printed to the standard output in normal36 If *quiet* is true, nothing is printed to the standard output in normal
35 operation.37 operation.
3638
3739 If *legacy* is true, old-style ``.pyc`` file path names are written,
38.. function:: compile_path(skip_curdir=True, maxlevels=0, force=False)40 otherwise (the default), :pep:`3147` style path names are written.
41
42
43.. function:: compile_path(skip_curdir=True, maxlevels=0, force=False, legacy=False)
3944
40 Byte-compile all the :file:`.py` files found along ``sys.path``. If45 Byte-compile all the :file:`.py` files found along ``sys.path``. If
41 *skip_curdir* is true (the default), the current directory is not included in46 *skip_curdir* is true (the default), the current directory is not included in
42 the search. The *maxlevels* and *force* parameters default to ``0`` and are47 the search. The *maxlevels* parameter defaults to ``0``, and the *force*
48 and *legacy* parameters default to ``False``. All are
43 passed to the :func:`compile_dir` function.49 passed to the :func:`compile_dir` function.
4450
45To force a recompile of all the :file:`.py` files in the :file:`Lib/`51To force a recompile of all the :file:`.py` files in the :file:`Lib/`
4652
=== modified file 'Doc/library/imp.rst'
--- Doc/library/imp.rst 2009-07-17 09:42:05 +0000
+++ Doc/library/imp.rst 2010-04-16 18:55:41 +0000
@@ -204,8 +204,41 @@
204 function does nothing.204 function does nothing.
205205
206206
207The following constants with integer values, defined in this module, are used to207The following functions and data provide conveniences for handling :pep:`3147`
208indicate the search result of :func:`find_module`.208byte-compiled file paths.
209
210.. versionadded:: 3.2
211
212.. function:: cache_from_source(path, debug_override=None)
213
214 Return the PEP 3147 path to the byte-compiled file associated with the
215 source *path*. For example, if *path* is ``/foo/bar/baz.py`` the return
216 value would be ``/foo/bar/__pycache__/baz.cpython-32.pyc`` for Python 3.2.
217 The ``cpython-32`` string comes from the current magic tag (see
218 :func:`get_tag`). The returned path will end in ``.pyc`` when
219 ``__debug__`` is True or ``.pyo`` for an optimized Python
220 (i.e. ``__debug__`` is False). By passing in True or False for
221 *debug_override* you can override the system's value for ``__debug__`` for
222 extension selection.
223
224 *path* need not exist.
225
226.. function:: source_from_cache(path)
227
228 Given the *path* to a PEP 3147 file name, return the associated source code
229 file path. For example, if *path* is
230 ``/foo/bar/__pycache__/baz.cpython-32.pyc`` the returned path would be
231 ``/foo/bar/baz.py``. *path* need not exist, however if it does not conform
232 to PEP 3147 format, a ``ValueError`` is raised.
233
234.. function:: get_tag()
235
236 Return the PEP 3147 magic tag string matching this version of Python's
237 magic number, as returned by :func:`get_magic`.
238
239
240The following constants with integer values, defined in this module, are used
241to indicate the search result of :func:`find_module`.
209242
210243
211.. data:: PY_SOURCE244.. data:: PY_SOURCE
212245
=== modified file 'Doc/library/py_compile.rst'
--- Doc/library/py_compile.rst 2009-09-02 19:34:52 +0000
+++ Doc/library/py_compile.rst 2010-04-16 18:55:41 +0000
@@ -26,12 +26,16 @@
2626
27 Compile a source file to byte-code and write out the byte-code cache file. The27 Compile a source file to byte-code and write out the byte-code cache file. The
28 source code is loaded from the file name *file*. The byte-code is written to28 source code is loaded from the file name *file*. The byte-code is written to
29 *cfile*, which defaults to *file* ``+`` ``'c'`` (``'o'`` if optimization is29 *cfile*, which defaults to the :PEP:`3147` path, ending in ``.pyc``
30 enabled in the current interpreter). If *dfile* is specified, it is used as the30 (``'.pyo`` if optimization is enabled in the current interpreter). For
31 example, if *file* is ``/foo/bar/baz.py`` *cfile* will default to
32 ``/foo/bar/__pycache__/baz.cpython-32.pyc`` for Python 3.2. If *dfile* is specified, it is used as the
31 name of the source file in error messages instead of *file*. If *doraise* is33 name of the source file in error messages instead of *file*. If *doraise* is
32 true, a :exc:`PyCompileError` is raised when an error is encountered while34 true, a :exc:`PyCompileError` is raised when an error is encountered while
33 compiling *file*. If *doraise* is false (the default), an error string is35 compiling *file*. If *doraise* is false (the default), an error string is
34 written to ``sys.stderr``, but no exception is raised.36 written to ``sys.stderr``, but no exception is raised. This function
37 returns the path to byte-compiled file, i.e. whatever *cfile* value was
38 used.
3539
3640
37.. function:: main(args=None)41.. function:: main(args=None)
3842
=== modified file 'Doc/library/runpy.rst'
--- Doc/library/runpy.rst 2010-01-04 09:29:10 +0000
+++ Doc/library/runpy.rst 2010-04-16 18:55:41 +0000
@@ -32,7 +32,8 @@
32 below are defined in the supplied dictionary, those definitions are32 below are defined in the supplied dictionary, those definitions are
33 overridden by :func:`run_module`.33 overridden by :func:`run_module`.
3434
35 The special global variables ``__name__``, ``__file__``, ``__loader__``35 The special global variables ``__name__``, ``__file__``, ``__cached__``,
36 ``__loader__``
36 and ``__package__`` are set in the globals dictionary before the module37 and ``__package__`` are set in the globals dictionary before the module
37 code is executed (Note that this is a minimal set of variables - other38 code is executed (Note that this is a minimal set of variables - other
38 variables may be set implicitly as an interpreter implementation detail).39 variables may be set implicitly as an interpreter implementation detail).
@@ -45,6 +46,8 @@
45 loader does not make filename information available, this variable is set46 loader does not make filename information available, this variable is set
46 to :const:`None`.47 to :const:`None`.
4748
49 ``__cached__`` will be set to ``None``.
50
48 ``__loader__`` is set to the PEP 302 module loader used to retrieve the51 ``__loader__`` is set to the PEP 302 module loader used to retrieve the
49 code for the module (This loader may be a wrapper around the standard52 code for the module (This loader may be a wrapper around the standard
50 import mechanism).53 import mechanism).
5154
=== modified file 'Include/import.h'
--- Include/import.h 2009-10-04 19:32:25 +0000
+++ Include/import.h 2010-04-16 18:55:41 +0000
@@ -8,9 +8,12 @@
8#endif8#endif
99
10PyAPI_FUNC(long) PyImport_GetMagicNumber(void);10PyAPI_FUNC(long) PyImport_GetMagicNumber(void);
11PyAPI_FUNC(const char *) PyImport_GetMagicTag(void);
11PyAPI_FUNC(PyObject *) PyImport_ExecCodeModule(char *name, PyObject *co);12PyAPI_FUNC(PyObject *) PyImport_ExecCodeModule(char *name, PyObject *co);
12PyAPI_FUNC(PyObject *) PyImport_ExecCodeModuleEx(13PyAPI_FUNC(PyObject *) PyImport_ExecCodeModuleEx(
13 char *name, PyObject *co, char *pathname);14 char *name, PyObject *co, char *pathname);
15PyAPI_FUNC(PyObject *) PyImport_ExecCodeModuleWithPathnames(
16 char *name, PyObject *co, char *pathname, char *cpathname);
14PyAPI_FUNC(PyObject *) PyImport_GetModuleDict(void);17PyAPI_FUNC(PyObject *) PyImport_GetModuleDict(void);
15PyAPI_FUNC(PyObject *) PyImport_AddModule(const char *name);18PyAPI_FUNC(PyObject *) PyImport_AddModule(const char *name);
16PyAPI_FUNC(PyObject *) PyImport_ImportModule(const char *name);19PyAPI_FUNC(PyObject *) PyImport_ImportModule(const char *name);
1720
=== modified file 'Lib/compileall.py'
--- Lib/compileall.py 2010-03-21 23:13:07 +0000
+++ Lib/compileall.py 2010-04-16 18:55:41 +0000
@@ -12,6 +12,7 @@
1212
13"""13"""
14import os14import os
15import errno
15import sys16import sys
16import py_compile17import py_compile
17import struct18import struct
@@ -20,7 +21,7 @@
20__all__ = ["compile_dir","compile_file","compile_path"]21__all__ = ["compile_dir","compile_file","compile_path"]
2122
22def compile_dir(dir, maxlevels=10, ddir=None,23def compile_dir(dir, maxlevels=10, ddir=None,
23 force=0, rx=None, quiet=0):24 force=False, rx=None, quiet=False, legacy=False):
24 """Byte-compile all modules in the given directory tree.25 """Byte-compile all modules in the given directory tree.
2526
26 Arguments (only dir is required):27 Arguments (only dir is required):
@@ -29,8 +30,9 @@
29 maxlevels: maximum recursion level (default 10)30 maxlevels: maximum recursion level (default 10)
30 ddir: if given, purported directory name (this is the31 ddir: if given, purported directory name (this is the
31 directory name that will show up in error messages)32 directory name that will show up in error messages)
32 force: if 1, force compilation, even if timestamps are up-to-date33 force: if True, force compilation, even if timestamps are up-to-date
33 quiet: if 1, be quiet during compilation34 quiet: if True, be quiet during compilation
35 legacy: if True, produce legacy pyc paths instead of PEP 3147 paths
3436
35 """37 """
36 if not quiet:38 if not quiet:
@@ -49,24 +51,26 @@
49 else:51 else:
50 dfile = None52 dfile = None
51 if not os.path.isdir(fullname):53 if not os.path.isdir(fullname):
52 if not compile_file(fullname, ddir, force, rx, quiet):54 if not compile_file(fullname, ddir, force, rx, quiet, legacy):
53 success = 055 success = 0
54 elif maxlevels > 0 and \56 elif maxlevels > 0 and \
55 name != os.curdir and name != os.pardir and \57 name != os.curdir and name != os.pardir and \
56 os.path.isdir(fullname) and \58 os.path.isdir(fullname) and \
57 not os.path.islink(fullname):59 not os.path.islink(fullname):
58 if not compile_dir(fullname, maxlevels - 1, dfile, force, rx,60 if not compile_dir(fullname, maxlevels - 1, dfile, force, rx,
59 quiet):61 quiet, legacy):
60 success = 062 success = 0
61 return success63 return success
6264
63def compile_file(fullname, ddir=None, force=0, rx=None, quiet=0):65def compile_file(fullname, ddir=None, force=0, rx=None, quiet=False,
66 legacy=False):
64 """Byte-compile file.67 """Byte-compile file.
65 file: the file to byte-compile68 fullname: the file to byte-compile
66 ddir: if given, purported directory name (this is the69 ddir: if given, purported directory name (this is the
67 directory name that will show up in error messages)70 directory name that will show up in error messages)
68 force: if 1, force compilation, even if timestamps are up-to-date71 force: if True, force compilation, even if timestamps are up-to-date
69 quiet: if 1, be quiet during compilation72 quiet: if True, be quiet during compilation
73 legacy: if True, produce legacy pyc paths instead of PEP 3147 paths
7074
71 """75 """
72 success = 176 success = 1
@@ -80,13 +84,22 @@
80 if mo:84 if mo:
81 return success85 return success
82 if os.path.isfile(fullname):86 if os.path.isfile(fullname):
87 if legacy:
88 cfile = fullname + ('c' if __debug__ else 'o')
89 else:
90 cfile = imp.cache_from_source(fullname)
91 cache_dir = os.path.dirname(cfile)
92 try:
93 os.mkdir(cache_dir)
94 except OSError as error:
95 if error.errno != errno.EEXIST:
96 raise
83 head, tail = name[:-3], name[-3:]97 head, tail = name[:-3], name[-3:]
84 if tail == '.py':98 if tail == '.py':
85 if not force:99 if not force:
86 try:100 try:
87 mtime = int(os.stat(fullname).st_mtime)101 mtime = int(os.stat(fullname).st_mtime)
88 expect = struct.pack('<4sl', imp.get_magic(), mtime)102 expect = struct.pack('<4sl', imp.get_magic(), mtime)
89 cfile = fullname + (__debug__ and 'c' or 'o')
90 with open(cfile, 'rb') as chandle:103 with open(cfile, 'rb') as chandle:
91 actual = chandle.read(8)104 actual = chandle.read(8)
92 if expect == actual:105 if expect == actual:
@@ -96,14 +109,15 @@
96 if not quiet:109 if not quiet:
97 print('Compiling', fullname, '...')110 print('Compiling', fullname, '...')
98 try:111 try:
99 ok = py_compile.compile(fullname, None, dfile, True)112 ok = py_compile.compile(fullname, cfile, dfile, True)
100 except py_compile.PyCompileError as err:113 except py_compile.PyCompileError as err:
101 if quiet:114 if quiet:
102 print('*** Error compiling', fullname, '...')115 print('*** Error compiling', fullname, '...')
103 else:116 else:
104 print('*** ', end='')117 print('*** ', end='')
105 # escape non-printable characters in msg118 # escape non-printable characters in msg
106 msg = err.msg.encode(sys.stdout.encoding, errors='backslashreplace')119 msg = err.msg.encode(sys.stdout.encoding,
120 errors='backslashreplace')
107 msg = msg.decode(sys.stdout.encoding)121 msg = msg.decode(sys.stdout.encoding)
108 print(msg)122 print(msg)
109 success = 0123 success = 0
@@ -119,15 +133,17 @@
119 success = 0133 success = 0
120 return success134 return success
121135
122def compile_path(skip_curdir=1, maxlevels=0, force=0, quiet=0):136def compile_path(skip_curdir=1, maxlevels=0, force=False, quiet=False,
137 legacy=False):
123 """Byte-compile all module on sys.path.138 """Byte-compile all module on sys.path.
124139
125 Arguments (all optional):140 Arguments (all optional):
126141
127 skip_curdir: if true, skip current directory (default true)142 skip_curdir: if true, skip current directory (default true)
128 maxlevels: max recursion level (default 0)143 maxlevels: max recursion level (default 0)
129 force: as for compile_dir() (default 0)144 force: as for compile_dir() (default False)
130 quiet: as for compile_dir() (default 0)145 quiet: as for compile_dir() (default False)
146 legacy: as for compile_dir() (default False)
131147
132 """148 """
133 success = 1149 success = 1
@@ -136,7 +152,8 @@
136 print('Skipping current directory')152 print('Skipping current directory')
137 else:153 else:
138 success = success and compile_dir(dir, maxlevels, None,154 success = success and compile_dir(dir, maxlevels, None,
139 force, quiet=quiet)155 force, quiet=quiet,
156 legacy=legacy)
140 return success157 return success
141158
142def expand_args(args, flist):159def expand_args(args, flist):
@@ -162,10 +179,10 @@
162 """Script main program."""179 """Script main program."""
163 import getopt180 import getopt
164 try:181 try:
165 opts, args = getopt.getopt(sys.argv[1:], 'lfqd:x:i:')182 opts, args = getopt.getopt(sys.argv[1:], 'lfqd:x:i:b')
166 except getopt.error as msg:183 except getopt.error as msg:
167 print(msg)184 print(msg)
168 print("usage: python compileall.py [-l] [-f] [-q] [-d destdir] " \185 print("usage: python compileall.py [-l] [-f] [-q] [-d destdir] "
169 "[-x regexp] [-i list] [directory|file ...]")186 "[-x regexp] [-i list] [directory|file ...]")
170 print("-l: don't recurse down")187 print("-l: don't recurse down")
171 print("-f: force rebuild even if timestamps are up-to-date")188 print("-f: force rebuild even if timestamps are up-to-date")
@@ -174,23 +191,27 @@
174 print(" if no directory arguments, -l sys.path is assumed")191 print(" if no directory arguments, -l sys.path is assumed")
175 print("-x regexp: skip files matching the regular expression regexp")192 print("-x regexp: skip files matching the regular expression regexp")
176 print(" the regexp is searched for in the full path of the file")193 print(" the regexp is searched for in the full path of the file")
177 print("-i list: expand list with its content (file and directory names)")194 print("-i list: expand list with its content "
195 "(file and directory names)")
196 print("-b: Produce legacy byte-compile file paths")
178 sys.exit(2)197 sys.exit(2)
179 maxlevels = 10198 maxlevels = 10
180 ddir = None199 ddir = None
181 force = 0200 force = False
182 quiet = 0201 quiet = False
183 rx = None202 rx = None
184 flist = None203 flist = None
204 legacy = False
185 for o, a in opts:205 for o, a in opts:
186 if o == '-l': maxlevels = 0206 if o == '-l': maxlevels = 0
187 if o == '-d': ddir = a207 if o == '-d': ddir = a
188 if o == '-f': force = 1208 if o == '-f': force = True
189 if o == '-q': quiet = 1209 if o == '-q': quiet = True
190 if o == '-x':210 if o == '-x':
191 import re211 import re
192 rx = re.compile(a)212 rx = re.compile(a)
193 if o == '-i': flist = a213 if o == '-i': flist = a
214 if o == '-b': legacy = True
194 if ddir:215 if ddir:
195 if len(args) != 1 and not os.path.isdir(args[0]):216 if len(args) != 1 and not os.path.isdir(args[0]):
196 print("-d destdir require exactly one directory argument")217 print("-d destdir require exactly one directory argument")
@@ -207,13 +228,14 @@
207 for arg in args:228 for arg in args:
208 if os.path.isdir(arg):229 if os.path.isdir(arg):
209 if not compile_dir(arg, maxlevels, ddir,230 if not compile_dir(arg, maxlevels, ddir,
210 force, rx, quiet):231 force, rx, quiet, legacy):
211 success = 0232 success = 0
212 else:233 else:
213 if not compile_file(arg, ddir, force, rx, quiet):234 if not compile_file(arg, ddir, force, rx,
235 quiet, legacy):
214 success = 0236 success = 0
215 else:237 else:
216 success = compile_path()238 success = compile_path(legacy=legacy)
217 except KeyboardInterrupt:239 except KeyboardInterrupt:
218 print("\n[interrupt]")240 print("\n[interrupt]")
219 success = 0241 success = 0
220242
=== modified file 'Lib/importlib/_bootstrap.py'
--- Lib/importlib/_bootstrap.py 2010-03-19 20:44:28 +0000
+++ Lib/importlib/_bootstrap.py 2010-04-16 18:55:41 +0000
@@ -488,6 +488,16 @@
488488
489 """Load a module from a source or bytecode file."""489 """Load a module from a source or bytecode file."""
490490
491 def _find_path(self, ext_type):
492 """Return PEP 3147 path if ext_type is PY_COMPILED, otherwise
493 super()._find_path() is called."""
494 if ext_type == imp.PY_COMPILED:
495 # We don't really care what the extension on self._base_path is,
496 # as long as it has exactly one dot.
497 bytecode_path = imp.cache_from_source(self._base_path + '.py')
498 return (bytecode_path if _path_exists(bytecode_path) else None)
499 return super()._find_path(ext_type)
500
491 @_check_name501 @_check_name
492 def source_mtime(self, name):502 def source_mtime(self, name):
493 """Return the modification time of the source for the specified503 """Return the modification time of the source for the specified
@@ -515,7 +525,16 @@
515 """525 """
516 bytecode_path = self.bytecode_path(name)526 bytecode_path = self.bytecode_path(name)
517 if not bytecode_path:527 if not bytecode_path:
518 bytecode_path = self._base_path + _suffix_list(imp.PY_COMPILED)[0]528 source_path = self.source_path(name)
529 bytecode_path = imp.cache_from_source(source_path)
530 # Ensure that the __pycache__ directory exists. We can't use
531 # os.path.dirname() here.
532 dirname, sep, basename = bytecode_path.rpartition(path_sep)
533 try:
534 _os.mkdir(dirname)
535 except OSError as error:
536 if error.errno != errno.EEXIST:
537 raise
519 try:538 try:
520 # Assuming bytes.539 # Assuming bytes.
521 with _closing(_io.FileIO(bytecode_path, 'w')) as bytecode_file:540 with _closing(_io.FileIO(bytecode_path, 'w')) as bytecode_file:
522541
=== modified file 'Lib/importlib/test/__main__.py'
--- Lib/importlib/test/__main__.py 2010-03-19 20:44:28 +0000
+++ Lib/importlib/test/__main__.py 2010-04-16 18:55:41 +0000
@@ -13,7 +13,12 @@
1313
1414
15def test_main():15def test_main():
16 start_dir = os.path.dirname(__file__)16 if '__pycache__' in __file__:
17 parts = __file__.split(os.path.sep)
18 start_dir = sep.join(parts[:-2])
19 else:
20 start_dir = os.path.dirname(__file__)
21 # XXX 2010-03-18 barry: Fix __file__
17 top_dir = os.path.dirname(os.path.dirname(start_dir))22 top_dir = os.path.dirname(os.path.dirname(start_dir))
18 test_loader = unittest.TestLoader()23 test_loader = unittest.TestLoader()
19 if '--builtin' in sys.argv:24 if '--builtin' in sys.argv:
2025
=== modified file 'Lib/importlib/test/source/test_file_loader.py'
--- Lib/importlib/test/source/test_file_loader.py 2010-03-19 20:44:28 +0000
+++ Lib/importlib/test/source/test_file_loader.py 2010-04-16 18:55:41 +0000
@@ -127,7 +127,7 @@
127 except KeyError:127 except KeyError:
128 pass128 pass
129 py_compile.compile(mapping[name])129 py_compile.compile(mapping[name])
130 bytecode_path = source_util.bytecode_path(mapping[name])130 bytecode_path = imp.cache_from_source(mapping[name])
131 with open(bytecode_path, 'rb') as file:131 with open(bytecode_path, 'rb') as file:
132 bc = file.read()132 bc = file.read()
133 new_bc = manipulator(bc)133 new_bc = manipulator(bc)
@@ -226,7 +226,7 @@
226 zeros = b'\x00\x00\x00\x00'226 zeros = b'\x00\x00\x00\x00'
227 with source_util.create_modules('_temp') as mapping:227 with source_util.create_modules('_temp') as mapping:
228 py_compile.compile(mapping['_temp'])228 py_compile.compile(mapping['_temp'])
229 bytecode_path = source_util.bytecode_path(mapping['_temp'])229 bytecode_path = imp.cache_from_source(mapping['_temp'])
230 with open(bytecode_path, 'r+b') as bytecode_file:230 with open(bytecode_path, 'r+b') as bytecode_file:
231 bytecode_file.seek(4)231 bytecode_file.seek(4)
232 bytecode_file.write(zeros)232 bytecode_file.write(zeros)
@@ -242,9 +242,10 @@
242 def test_bad_marshal(self):242 def test_bad_marshal(self):
243 # Bad marshal data should raise a ValueError.243 # Bad marshal data should raise a ValueError.
244 with source_util.create_modules('_temp') as mapping:244 with source_util.create_modules('_temp') as mapping:
245 bytecode_path = source_util.bytecode_path(mapping['_temp'])245 bytecode_path = imp.cache_from_source(mapping['_temp'])
246 source_mtime = os.path.getmtime(mapping['_temp'])246 source_mtime = os.path.getmtime(mapping['_temp'])
247 source_timestamp = importlib._w_long(source_mtime)247 source_timestamp = importlib._w_long(source_mtime)
248 source_util.ensure_bytecode_path(bytecode_path)
248 with open(bytecode_path, 'wb') as bytecode_file:249 with open(bytecode_path, 'wb') as bytecode_file:
249 bytecode_file.write(imp.get_magic())250 bytecode_file.write(imp.get_magic())
250 bytecode_file.write(source_timestamp)251 bytecode_file.write(source_timestamp)
@@ -260,7 +261,7 @@
260 with source_util.create_modules('_temp') as mapping:261 with source_util.create_modules('_temp') as mapping:
261 # Create bytecode that will need to be re-created.262 # Create bytecode that will need to be re-created.
262 py_compile.compile(mapping['_temp'])263 py_compile.compile(mapping['_temp'])
263 bytecode_path = source_util.bytecode_path(mapping['_temp'])264 bytecode_path = imp.cache_from_source(mapping['_temp'])
264 with open(bytecode_path, 'r+b') as bytecode_file:265 with open(bytecode_path, 'r+b') as bytecode_file:
265 bytecode_file.seek(0)266 bytecode_file.seek(0)
266 bytecode_file.write(b'\x00\x00\x00\x00')267 bytecode_file.write(b'\x00\x00\x00\x00')
267268
=== modified file 'Lib/importlib/test/source/test_finder.py'
--- Lib/importlib/test/source/test_finder.py 2010-03-19 20:44:28 +0000
+++ Lib/importlib/test/source/test_finder.py 2010-04-16 18:55:41 +0000
@@ -1,7 +1,9 @@
1from importlib import _bootstrap1from importlib import _bootstrap
2from .. import abc2from .. import abc
3from . import util as source_util3from . import util as source_util
4from test.support import make_legacy_pyc
4import os5import os
6import errno
5import py_compile7import py_compile
6import unittest8import unittest
7import warnings9import warnings
@@ -52,6 +54,14 @@
52 if unlink:54 if unlink:
53 for name in unlink:55 for name in unlink:
54 os.unlink(mapping[name])56 os.unlink(mapping[name])
57 try:
58 make_legacy_pyc(mapping[name])
59 except OSError as error:
60 # Some tests do not set compile_=True so the source
61 # module will not get compiled and there will be no
62 # PEP 3147 pyc file to rename.
63 if error.errno != errno.ENOENT:
64 raise
55 loader = self.import_(mapping['.root'], test)65 loader = self.import_(mapping['.root'], test)
56 self.assertTrue(hasattr(loader, 'load_module'))66 self.assertTrue(hasattr(loader, 'load_module'))
57 return loader67 return loader
@@ -60,7 +70,8 @@
60 # [top-level source]70 # [top-level source]
61 self.run_test('top_level')71 self.run_test('top_level')
62 # [top-level bc]72 # [top-level bc]
63 self.run_test('top_level', compile_={'top_level'}, unlink={'top_level'})73 self.run_test('top_level', compile_={'top_level'},
74 unlink={'top_level'})
64 # [top-level both]75 # [top-level both]
65 self.run_test('top_level', compile_={'top_level'})76 self.run_test('top_level', compile_={'top_level'})
6677
6778
=== modified file 'Lib/importlib/test/source/test_source_encoding.py'
--- Lib/importlib/test/source/test_source_encoding.py 2010-03-19 20:44:28 +0000
+++ Lib/importlib/test/source/test_source_encoding.py 2010-04-16 18:55:41 +0000
@@ -33,7 +33,7 @@
3333
34 def run_test(self, source):34 def run_test(self, source):
35 with source_util.create_modules(self.module_name) as mapping:35 with source_util.create_modules(self.module_name) as mapping:
36 with open(mapping[self.module_name], 'wb')as file:36 with open(mapping[self.module_name], 'wb') as file:
37 file.write(source)37 file.write(source)
38 loader = _bootstrap._PyPycFileLoader(self.module_name,38 loader = _bootstrap._PyPycFileLoader(self.module_name,
39 mapping[self.module_name], False)39 mapping[self.module_name], False)
4040
=== modified file 'Lib/importlib/test/source/util.py'
--- Lib/importlib/test/source/util.py 2010-03-19 20:44:28 +0000
+++ Lib/importlib/test/source/util.py 2010-04-16 18:55:41 +0000
@@ -1,5 +1,6 @@
1from .. import util1from .. import util
2import contextlib2import contextlib
3import errno
3import functools4import functools
4import imp5import imp
5import os6import os
@@ -26,14 +27,16 @@
26 return wrapper27 return wrapper
2728
2829
29def bytecode_path(source_path):30def ensure_bytecode_path(bytecode_path):
30 for suffix, _, type_ in imp.get_suffixes():31 """Ensure that the __pycache__ directory for PEP 3147 pyc file exists.
31 if type_ == imp.PY_COMPILED:32
32 bc_suffix = suffix33 :param bytecode_path: File system path to PEP 3147 pyc file.
33 break34 """
34 else:35 try:
35 raise ValueError("no bytecode suffix is defined")36 os.mkdir(os.path.dirname(bytecode_path))
36 return os.path.splitext(source_path)[0] + bc_suffix37 except OSError as error:
38 if error.errno != errno.EEXIST:
39 raise
3740
3841
39@contextlib.contextmanager42@contextlib.contextmanager
4043
=== modified file 'Lib/importlib/util.py'
--- Lib/importlib/util.py 2010-03-19 20:44:28 +0000
+++ Lib/importlib/util.py 2010-04-16 18:55:41 +0000
@@ -1,4 +1,5 @@
1"""Utility code for constructing importers, etc."""1"""Utility code for constructing importers, etc."""
2
2from ._bootstrap import module_for_loader3from ._bootstrap import module_for_loader
3from ._bootstrap import set_loader4from ._bootstrap import set_loader
4from ._bootstrap import set_package5from ._bootstrap import set_package
56
=== modified file 'Lib/inspect.py'
--- Lib/inspect.py 2010-03-30 17:42:32 +0000
+++ Lib/inspect.py 2010-04-16 18:55:41 +0000
@@ -54,6 +54,7 @@
54 """Return true if the object is a module.54 """Return true if the object is a module.
5555
56 Module objects provide these attributes:56 Module objects provide these attributes:
57 __cached__ pathname to byte compiled file
57 __doc__ documentation string58 __doc__ documentation string
58 __file__ filename (missing for built-in modules)"""59 __file__ filename (missing for built-in modules)"""
59 return isinstance(object, types.ModuleType)60 return isinstance(object, types.ModuleType)
6061
=== modified file 'Lib/py_compile.py'
--- Lib/py_compile.py 2010-03-31 20:36:22 +0000
+++ Lib/py_compile.py 2010-04-16 18:55:41 +0000
@@ -4,6 +4,7 @@
4"""4"""
55
6import builtins6import builtins
7import errno
7import imp8import imp
8import marshal9import marshal
9import os10import os
@@ -37,16 +38,18 @@
37 can be accesses as class variable 'file'38 can be accesses as class variable 'file'
3839
39 msg: string message to be written as error message40 msg: string message to be written as error message
40 If no value is given, a default exception message will be given,41 If no value is given, a default exception message will be
41 consistent with 'standard' py_compile output.42 given, consistent with 'standard' py_compile output.
42 message (or default) can be accesses as class variable 'msg'43 message (or default) can be accesses as class variable
44 'msg'
4345
44 """46 """
4547
46 def __init__(self, exc_type, exc_value, file, msg=''):48 def __init__(self, exc_type, exc_value, file, msg=''):
47 exc_type_name = exc_type.__name__49 exc_type_name = exc_type.__name__
48 if exc_type is SyntaxError:50 if exc_type is SyntaxError:
49 tbtext = ''.join(traceback.format_exception_only(exc_type, exc_value))51 tbtext = ''.join(traceback.format_exception_only(
52 exc_type, exc_value))
50 errmsg = tbtext.replace('File "<string>"', 'File "%s"' % file)53 errmsg = tbtext.replace('File "<string>"', 'File "%s"' % file)
51 else:54 else:
52 errmsg = "Sorry: %s: %s" % (exc_type_name,exc_value)55 errmsg = "Sorry: %s: %s" % (exc_type_name,exc_value)
@@ -64,7 +67,7 @@
6467
65def wr_long(f, x):68def wr_long(f, x):
66 """Internal; write a 32-bit int to a file in little-endian order."""69 """Internal; write a 32-bit int to a file in little-endian order."""
67 f.write(bytes([x & 0xff,70 f.write(bytes([x & 0xff,
68 (x >> 8) & 0xff,71 (x >> 8) & 0xff,
69 (x >> 16) & 0xff,72 (x >> 16) & 0xff,
70 (x >> 24) & 0xff]))73 (x >> 24) & 0xff]))
@@ -72,20 +75,18 @@
72def compile(file, cfile=None, dfile=None, doraise=False):75def compile(file, cfile=None, dfile=None, doraise=False):
73 """Byte-compile one Python source file to Python bytecode.76 """Byte-compile one Python source file to Python bytecode.
7477
75 Arguments:78 :param file: The source file name.
7679 :param cfile: The target byte compiled file name. When not given, this
77 file: source filename80 defaults to the PEP 3147 location.
78 cfile: target filename; defaults to source with 'c' or 'o' appended81 :param dfile: Purported file name, i.e. the file name that shows up in
79 ('c' normally, 'o' in optimizing mode, giving .pyc or .pyo)82 error messages. Defaults to the source file name.
80 dfile: purported filename; defaults to source (this is the filename83 :param doraise: Flag indicating whether or not an exception should be
81 that will show up in error messages)84 raised when a compile error is found. If an exception occurs and this
82 doraise: flag indicating whether or not an exception should be85 flag is set to False, a string indicating the nature of the exception
83 raised when a compile error is found. If an exception86 will be printed, and the function will return to the caller. If an
84 occurs and this flag is set to False, a string87 exception occurs and this flag is set to True, a PyCompileError
85 indicating the nature of the exception will be printed,88 exception will be raised.
86 and the function will return to the caller. If an89 :return: Path to the resulting byte compiled file.
87 exception occurs and this flag is set to True, a
88 PyCompileError exception will be raised.
8990
90 Note that it isn't necessary to byte-compile Python modules for91 Note that it isn't necessary to byte-compile Python modules for
91 execution efficiency -- Python itself byte-compiles a module when92 execution efficiency -- Python itself byte-compiles a module when
@@ -102,7 +103,6 @@
102 See compileall.py for a script/module that uses this module to103 See compileall.py for a script/module that uses this module to
103 byte-compile all installed files (or all files in selected104 byte-compile all installed files (or all files in selected
104 directories).105 directories).
105
106 """106 """
107 with open(file, "rb") as f:107 with open(file, "rb") as f:
108 encoding = tokenize.detect_encoding(f.readline)[0]108 encoding = tokenize.detect_encoding(f.readline)[0]
@@ -122,7 +122,12 @@
122 sys.stderr.write(py_exc.msg + '\n')122 sys.stderr.write(py_exc.msg + '\n')
123 return123 return
124 if cfile is None:124 if cfile is None:
125 cfile = file + (__debug__ and 'c' or 'o')125 cfile = imp.cache_from_source(file)
126 try:
127 os.mkdir(os.path.dirname(cfile))
128 except OSError as error:
129 if error.errno != errno.EEXIST:
130 raise
126 with open(cfile, 'wb') as fc:131 with open(cfile, 'wb') as fc:
127 fc.write(b'\0\0\0\0')132 fc.write(b'\0\0\0\0')
128 wr_long(fc, timestamp)133 wr_long(fc, timestamp)
@@ -130,6 +135,7 @@
130 fc.flush()135 fc.flush()
131 fc.seek(0, 0)136 fc.seek(0, 0)
132 fc.write(MAGIC)137 fc.write(MAGIC)
138 return cfile
133139
134def main(args=None):140def main(args=None):
135 """Compile several source files.141 """Compile several source files.
136142
=== modified file 'Lib/pydoc.py'
--- Lib/pydoc.py 2010-04-01 03:05:25 +0000
+++ Lib/pydoc.py 2010-04-16 18:55:41 +0000
@@ -159,7 +159,8 @@
159 """Decide whether to show documentation on a variable."""159 """Decide whether to show documentation on a variable."""
160 # Certain special names are redundant.160 # Certain special names are redundant.
161 _hidden_names = ('__builtins__', '__doc__', '__file__', '__path__',161 _hidden_names = ('__builtins__', '__doc__', '__file__', '__path__',
162 '__module__', '__name__', '__slots__', '__package__')162 '__module__', '__name__', '__slots__', '__package__',
163 '__cached__')
163 if name in _hidden_names: return 0164 if name in _hidden_names: return 0
164 # Private names are hidden, but special names are displayed.165 # Private names are hidden, but special names are displayed.
165 if name.startswith('__') and name.endswith('__'): return 1166 if name.startswith('__') and name.endswith('__'): return 1
166167
=== modified file 'Lib/runpy.py'
--- Lib/runpy.py 2009-12-20 17:28:31 +0000
+++ Lib/runpy.py 2010-04-16 18:55:41 +0000
@@ -67,6 +67,7 @@
67 run_globals.update(init_globals)67 run_globals.update(init_globals)
68 run_globals.update(__name__ = mod_name,68 run_globals.update(__name__ = mod_name,
69 __file__ = mod_fname,69 __file__ = mod_fname,
70 __cached__ = None,
70 __loader__ = mod_loader,71 __loader__ = mod_loader,
71 __package__ = pkg_name)72 __package__ = pkg_name)
72 exec(code, run_globals)73 exec(code, run_globals)
@@ -130,6 +131,7 @@
130 At the very least, these variables in __main__ will be overwritten:131 At the very least, these variables in __main__ will be overwritten:
131 __name__132 __name__
132 __file__133 __file__
134 __cached__
133 __loader__135 __loader__
134 __package__136 __package__
135 """137 """
136138
=== modified file 'Lib/site.py'
--- Lib/site.py 2010-04-12 22:33:42 +0000
+++ Lib/site.py 2010-04-16 18:55:41 +0000
@@ -74,15 +74,19 @@
74 return dir, os.path.normcase(dir)74 return dir, os.path.normcase(dir)
7575
7676
77def abs__file__():77def abs_paths():
78 """Set all module' __file__ attribute to an absolute path"""78 """Set all module __file__ and __cached__ attributes to an absolute path"""
79 for m in set(sys.modules.values()):79 for m in set(sys.modules.values()):
80 if hasattr(m, '__loader__'):80 if hasattr(m, '__loader__'):
81 continue # don't mess with a PEP 302-supplied __file__81 continue # don't mess with a PEP 302-supplied __file__
82 try:82 try:
83 m.__file__ = os.path.abspath(m.__file__)83 m.__file__ = os.path.abspath(m.__file__)
84 except AttributeError:84 except AttributeError:
85 continue85 pass
86 try:
87 m.__cached__ = os.path.abspath(m.__cached__)
88 except AttributeError:
89 pass
8690
8791
88def removeduppaths():92def removeduppaths():
@@ -518,7 +522,7 @@
518def main():522def main():
519 global ENABLE_USER_SITE523 global ENABLE_USER_SITE
520524
521 abs__file__()525 abs_paths()
522 known_paths = removeduppaths()526 known_paths = removeduppaths()
523 if (os.name == "posix" and sys.path and527 if (os.name == "posix" and sys.path and
524 os.path.basename(sys.path[-1]) == "Modules"):528 os.path.basename(sys.path[-1]) == "Modules"):
525529
=== modified file 'Lib/test/script_helper.py'
--- Lib/test/script_helper.py 2010-02-27 16:12:22 +0000
+++ Lib/test/script_helper.py 2010-04-16 18:55:41 +0000
@@ -11,6 +11,9 @@
11import shutil11import shutil
12import zipfile12import zipfile
1313
14from imp import source_from_cache
15from test.support import make_legacy_pyc
16
14# Executing the interpreter in a subprocess17# Executing the interpreter in a subprocess
15def python_exit_code(*args):18def python_exit_code(*args):
16 cmd_line = [sys.executable, '-E']19 cmd_line = [sys.executable, '-E']
@@ -62,20 +65,18 @@
62 script_file.close()65 script_file.close()
63 return script_name66 return script_name
6467
65def compile_script(script_name):
66 py_compile.compile(script_name, doraise=True)
67 if __debug__:
68 compiled_name = script_name + 'c'
69 else:
70 compiled_name = script_name + 'o'
71 return compiled_name
72
73def make_zip_script(zip_dir, zip_basename, script_name, name_in_zip=None):68def make_zip_script(zip_dir, zip_basename, script_name, name_in_zip=None):
74 zip_filename = zip_basename+os.extsep+'zip'69 zip_filename = zip_basename+os.extsep+'zip'
75 zip_name = os.path.join(zip_dir, zip_filename)70 zip_name = os.path.join(zip_dir, zip_filename)
76 zip_file = zipfile.ZipFile(zip_name, 'w')71 zip_file = zipfile.ZipFile(zip_name, 'w')
77 if name_in_zip is None:72 if name_in_zip is None:
78 name_in_zip = os.path.basename(script_name)73 parts = script_name.split(os.sep)
74 if len(parts) >= 2 and parts[-2] == '__pycache__':
75 legacy_pyc = make_legacy_pyc(source_from_cache(script_name))
76 name_in_zip = os.path.basename(legacy_pyc)
77 script_name = legacy_pyc
78 else:
79 name_in_zip = os.path.basename(script_name)
79 zip_file.write(script_name, name_in_zip)80 zip_file.write(script_name, name_in_zip)
80 zip_file.close()81 zip_file.close()
81 #if test.test_support.verbose:82 #if test.test_support.verbose:
@@ -98,8 +99,8 @@
98 script_name = make_script(zip_dir, script_basename, source)99 script_name = make_script(zip_dir, script_basename, source)
99 unlink.append(script_name)100 unlink.append(script_name)
100 if compiled:101 if compiled:
101 init_name = compile_script(init_name)102 init_name = py_compile(init_name, doraise=True)
102 script_name = compile_script(script_name)103 script_name = py_compile(script_name, doraise=True)
103 unlink.extend((init_name, script_name))104 unlink.extend((init_name, script_name))
104 pkg_names = [os.sep.join([pkg_name]*i) for i in range(1, depth+1)]105 pkg_names = [os.sep.join([pkg_name]*i) for i in range(1, depth+1)]
105 script_name_in_zip = os.path.join(pkg_names[-1], os.path.basename(script_name))106 script_name_in_zip = os.path.join(pkg_names[-1], os.path.basename(script_name))
106107
=== modified file 'Lib/test/support.py'
--- Lib/test/support.py 2010-04-11 20:22:10 +0000
+++ Lib/test/support.py 2010-04-16 18:55:41 +0000
@@ -17,22 +17,25 @@
17import importlib17import importlib
18import collections18import collections
19import re19import re
20import imp
20import time21import time
2122
22__all__ = ["Error", "TestFailed", "ResourceDenied", "import_module",23__all__ = [
23 "verbose", "use_resources", "max_memuse", "record_original_stdout",24 "Error", "TestFailed", "ResourceDenied", "import_module",
24 "get_original_stdout", "unload", "unlink", "rmtree", "forget",25 "verbose", "use_resources", "max_memuse", "record_original_stdout",
25 "is_resource_enabled", "requires", "find_unused_port", "bind_port",26 "get_original_stdout", "unload", "unlink", "rmtree", "forget",
26 "fcmp", "is_jython", "TESTFN", "HOST", "FUZZ", "SAVEDCWD", "temp_cwd",27 "is_resource_enabled", "requires", "find_unused_port", "bind_port",
27 "findfile", "sortdict", "check_syntax_error", "open_urlresource",28 "fcmp", "is_jython", "TESTFN", "HOST", "FUZZ", "SAVEDCWD", "temp_cwd",
28 "check_warnings", "CleanImport", "EnvironmentVarGuard",29 "findfile", "sortdict", "check_syntax_error", "open_urlresource",
29 "TransientResource", "captured_output", "captured_stdout",30 "check_warnings", "CleanImport", "EnvironmentVarGuard",
30 "time_out", "socket_peer_reset", "ioerror_peer_reset",31 "TransientResource", "captured_output", "captured_stdout",
31 "run_with_locale",32 "time_out", "socket_peer_reset", "ioerror_peer_reset",
32 "set_memlimit", "bigmemtest", "bigaddrspacetest", "BasicTestRunner",33 "run_with_locale", 'temp_umask',
33 "run_unittest", "run_doctest", "threading_setup", "threading_cleanup",34 "set_memlimit", "bigmemtest", "bigaddrspacetest", "BasicTestRunner",
34 "reap_children", "cpython_only", "check_impl_detail", "get_attribute",35 "run_unittest", "run_doctest", "threading_setup", "threading_cleanup",
35 "swap_item", "swap_attr"]36 "reap_children", "cpython_only", "check_impl_detail", "get_attribute",
37 "swap_item", "swap_attr",
38 ]
3639
3740
38class Error(Exception):41class Error(Exception):
@@ -177,27 +180,50 @@
177def unlink(filename):180def unlink(filename):
178 try:181 try:
179 os.unlink(filename)182 os.unlink(filename)
180 except OSError:183 except OSError as error:
181 pass184 # The filename need not exist.
185 if error.errno != errno.ENOENT:
186 raise
182187
183def rmtree(path):188def rmtree(path):
184 try:189 try:
185 shutil.rmtree(path)190 shutil.rmtree(path)
186 except OSError as e:191 except OSError as error:
187 # Unix returns ENOENT, Windows returns ESRCH.192 # Unix returns ENOENT, Windows returns ESRCH.
188 if e.errno not in (errno.ENOENT, errno.ESRCH):193 if error.errno not in (errno.ENOENT, errno.ESRCH):
189 raise194 raise
190195
196def make_legacy_pyc(source):
197 """Move a PEP 3147 pyc/pyo file to its legacy pyc/pyo location.
198
199 The choice of .pyc or .pyo extension is done based on the __debug__ flag
200 value.
201
202 :param source: The file system path to the source file. The source file
203 does not need to exist, however the PEP 3147 pyc file must exist.
204 :return: The file system path to the legacy pyc file.
205 """
206 pyc_file = imp.cache_from_source(source)
207 up_one = os.path.dirname(os.path.abspath(source))
208 legacy_pyc = os.path.join(up_one, source + ('c' if __debug__ else 'o'))
209 os.rename(pyc_file, legacy_pyc)
210 return legacy_pyc
211
191def forget(modname):212def forget(modname):
192 '''"Forget" a module was ever imported by removing it from sys.modules and213 """'Forget' a module was ever imported.
193 deleting any .pyc and .pyo files.'''214
215 This removes the module from sys.modules and deletes any PEP 3147 or
216 legacy .pyc and .pyo files.
217 """
194 unload(modname)218 unload(modname)
195 for dirname in sys.path:219 for dirname in sys.path:
196 unlink(os.path.join(dirname, modname + '.pyc'))220 source = os.path.join(dirname, modname + '.py')
197 # Deleting the .pyo file cannot be within the 'try' for the .pyc since221 # It doesn't matter if they exist or not, unlink all possible
198 # the chance exists that there is no .pyc (and thus the 'try' statement222 # combinations of PEP 3147 and legacy pyc and pyo files.
199 # is exited) but there is a .pyo file.223 unlink(source + 'c')
200 unlink(os.path.join(dirname, modname + '.pyo'))224 unlink(source + 'o')
225 unlink(imp.cache_from_source(source, debug_override=True))
226 unlink(imp.cache_from_source(source, debug_override=False))
201227
202def is_resource_enabled(resource):228def is_resource_enabled(resource):
203 """Test whether a resource is enabled. Known resources are set by229 """Test whether a resource is enabled. Known resources are set by
@@ -208,7 +234,9 @@
208 """Raise ResourceDenied if the specified resource is not available.234 """Raise ResourceDenied if the specified resource is not available.
209235
210 If the caller's module is __main__ then automatically return True. The236 If the caller's module is __main__ then automatically return True. The
211 possibility of False being returned occurs when regrtest.py is executing."""237 possibility of False being returned occurs when regrtest.py is
238 executing.
239 """
212 # see if the caller's module is __main__ - if so, treat as if240 # see if the caller's module is __main__ - if so, treat as if
213 # the resource was set241 # the resource was set
214 if sys._getframe(1).f_globals.get("__name__") == "__main__":242 if sys._getframe(1).f_globals.get("__name__") == "__main__":
@@ -405,6 +433,16 @@
405 rmtree(name)433 rmtree(name)
406434
407435
436@contextlib.contextmanager
437def temp_umask(umask):
438 """Context manager that temporarily sets the process umask."""
439 oldmask = os.umask(umask)
440 try:
441 yield
442 finally:
443 os.umask(oldmask)
444
445
408def findfile(file, here=__file__, subdir=None):446def findfile(file, here=__file__, subdir=None):
409 """Try to find a file on sys.path and the working directory. If it is not447 """Try to find a file on sys.path and the working directory. If it is not
410 found the argument passed to the function is returned (this does not448 found the argument passed to the function is returned (this does not
411449
=== modified file 'Lib/test/test_cmd_line_script.py'
--- Lib/test/test_cmd_line_script.py 2010-03-14 10:45:50 +0000
+++ Lib/test/test_cmd_line_script.py 2010-04-16 18:55:41 +0000
@@ -1,12 +1,14 @@
1# Tests command line execution of scripts1# tests command line execution of scripts
22
3import unittest3import unittest
4import os4import os
5import os.path5import os.path
6import py_compile
7
6import test.support8import test.support
7from test.script_helper import (run_python,9from test.script_helper import (
8 temp_dir, make_script, compile_script,10 make_pkg, make_script, make_zip_pkg, make_zip_script, run_python,
9 make_pkg, make_zip_script, make_zip_pkg)11 temp_dir)
1012
11verbose = test.support.verbose13verbose = test.support.verbose
1214
@@ -28,6 +30,7 @@
28# Check population of magic variables30# Check population of magic variables
29assertEqual(__name__, '__main__')31assertEqual(__name__, '__main__')
30print('__file__==%r' % __file__)32print('__file__==%r' % __file__)
33assertEqual(__cached__, None)
31print('__package__==%r' % __package__)34print('__package__==%r' % __package__)
32# Check the sys module35# Check the sys module
33import sys36import sys
@@ -101,9 +104,10 @@
101 def test_script_compiled(self):104 def test_script_compiled(self):
102 with temp_dir() as script_dir:105 with temp_dir() as script_dir:
103 script_name = _make_test_script(script_dir, 'script')106 script_name = _make_test_script(script_dir, 'script')
104 compiled_name = compile_script(script_name)107 compiled_name = py_compile.compile(script_name, doraise=True)
105 os.remove(script_name)108 os.remove(script_name)
106 self._check_script(compiled_name, compiled_name, compiled_name, None)109 self._check_script(compiled_name, compiled_name,
110 compiled_name, None)
107111
108 def test_directory(self):112 def test_directory(self):
109 with temp_dir() as script_dir:113 with temp_dir() as script_dir:
@@ -113,9 +117,10 @@
113 def test_directory_compiled(self):117 def test_directory_compiled(self):
114 with temp_dir() as script_dir:118 with temp_dir() as script_dir:
115 script_name = _make_test_script(script_dir, '__main__')119 script_name = _make_test_script(script_dir, '__main__')
116 compiled_name = compile_script(script_name)120 compiled_name = py_compile.compile(script_name, doraise=True)
117 os.remove(script_name)121 os.remove(script_name)
118 self._check_script(script_dir, compiled_name, script_dir, '')122 pyc_file = test.support.make_legacy_pyc(script_name)
123 self._check_script(script_dir, pyc_file, script_dir, '')
119124
120 def test_directory_error(self):125 def test_directory_error(self):
121 with temp_dir() as script_dir:126 with temp_dir() as script_dir:
@@ -131,7 +136,7 @@
131 def test_zipfile_compiled(self):136 def test_zipfile_compiled(self):
132 with temp_dir() as script_dir:137 with temp_dir() as script_dir:
133 script_name = _make_test_script(script_dir, '__main__')138 script_name = _make_test_script(script_dir, '__main__')
134 compiled_name = compile_script(script_name)139 compiled_name = py_compile.compile(script_name, doraise=True)
135 zip_name, run_name = make_zip_script(script_dir, 'test_zip', compiled_name)140 zip_name, run_name = make_zip_script(script_dir, 'test_zip', compiled_name)
136 self._check_script(zip_name, run_name, zip_name, '')141 self._check_script(zip_name, run_name, zip_name, '')
137142
@@ -176,11 +181,12 @@
176 pkg_dir = os.path.join(script_dir, 'test_pkg')181 pkg_dir = os.path.join(script_dir, 'test_pkg')
177 make_pkg(pkg_dir)182 make_pkg(pkg_dir)
178 script_name = _make_test_script(pkg_dir, '__main__')183 script_name = _make_test_script(pkg_dir, '__main__')
179 compiled_name = compile_script(script_name)184 compiled_name = py_compile.compile(script_name, doraise=True)
180 os.remove(script_name)185 os.remove(script_name)
186 pyc_file = test.support.make_legacy_pyc(script_name)
181 launch_name = _make_launch_script(script_dir, 'launch', 'test_pkg')187 launch_name = _make_launch_script(script_dir, 'launch', 'test_pkg')
182 self._check_script(launch_name, compiled_name,188 self._check_script(launch_name, pyc_file,
183 compiled_name, 'test_pkg')189 pyc_file, 'test_pkg')
184190
185 def test_package_error(self):191 def test_package_error(self):
186 with temp_dir() as script_dir:192 with temp_dir() as script_dir:
187193
=== modified file 'Lib/test/test_compileall.py'
--- Lib/test/test_compileall.py 2010-03-16 13:19:21 +0000
+++ Lib/test/test_compileall.py 2010-04-16 18:55:41 +0000
@@ -5,22 +5,23 @@
5import py_compile5import py_compile
6import shutil6import shutil
7import struct7import struct
8import subprocess
8import tempfile9import tempfile
9from test import support
10import unittest10import unittest
11import io11import io
1212
13from test import support
1314
14class CompileallTests(unittest.TestCase):15class CompileallTests(unittest.TestCase):
1516
16 def setUp(self):17 def setUp(self):
17 self.directory = tempfile.mkdtemp()18 self.directory = tempfile.mkdtemp()
18 self.source_path = os.path.join(self.directory, '_test.py')19 self.source_path = os.path.join(self.directory, '_test.py')
19 self.bc_path = self.source_path + ('c' if __debug__ else 'o')20 self.bc_path = imp.cache_from_source(self.source_path)
20 with open(self.source_path, 'w') as file:21 with open(self.source_path, 'w') as file:
21 file.write('x = 123\n')22 file.write('x = 123\n')
22 self.source_path2 = os.path.join(self.directory, '_test2.py')23 self.source_path2 = os.path.join(self.directory, '_test2.py')
23 self.bc_path2 = self.source_path2 + ('c' if __debug__ else 'o')24 self.bc_path2 = imp.cache_from_source(self.source_path2)
24 shutil.copyfile(self.source_path, self.source_path2)25 shutil.copyfile(self.source_path, self.source_path2)
2526
26 def tearDown(self):27 def tearDown(self):
@@ -65,17 +66,19 @@
65 except:66 except:
66 pass67 pass
67 compileall.compile_file(self.source_path, force=False, quiet=True)68 compileall.compile_file(self.source_path, force=False, quiet=True)
68 self.assertTrue(os.path.isfile(self.bc_path) \69 self.assertTrue(os.path.isfile(self.bc_path) and
69 and not os.path.isfile(self.bc_path2))70 not os.path.isfile(self.bc_path2))
70 os.unlink(self.bc_path)71 os.unlink(self.bc_path)
71 compileall.compile_dir(self.directory, force=False, quiet=True)72 compileall.compile_dir(self.directory, force=False, quiet=True)
72 self.assertTrue(os.path.isfile(self.bc_path) \73 self.assertTrue(os.path.isfile(self.bc_path) and
73 and os.path.isfile(self.bc_path2))74 os.path.isfile(self.bc_path2))
74 os.unlink(self.bc_path)75 os.unlink(self.bc_path)
75 os.unlink(self.bc_path2)76 os.unlink(self.bc_path2)
7677
78
77class EncodingTest(unittest.TestCase):79class EncodingTest(unittest.TestCase):
78 'Issue 6716: compileall should escape source code when printing errors to stdout.'80 """Issue 6716: compileall should escape source code when printing errors
81 to stdout."""
7982
80 def setUp(self):83 def setUp(self):
81 self.directory = tempfile.mkdtemp()84 self.directory = tempfile.mkdtemp()
@@ -95,9 +98,65 @@
95 finally:98 finally:
96 sys.stdout = orig_stdout99 sys.stdout = orig_stdout
97100
101class CommandLineTests(unittest.TestCase):
102 """Test some aspects of compileall's CLI."""
103
104 def setUp(self):
105 self.addCleanup(self._cleanup)
106 self.directory = tempfile.mkdtemp()
107 self.pkgdir = os.path.join(self.directory, 'foo')
108 os.mkdir(self.pkgdir)
109 # Touch the __init__.py and a package module.
110 with open(os.path.join(self.pkgdir, '__init__.py'), 'w'):
111 pass
112 with open(os.path.join(self.pkgdir, 'bar.py'), 'w'):
113 pass
114 sys.path.insert(0, self.directory)
115
116 def _cleanup(self):
117 support.rmtree(self.directory)
118 assert sys.path[0] == self.directory, 'Missing path'
119 del sys.path[0]
120
121 def test_pep3147_paths(self):
122 # Ensure that the default behavior of compileall's CLI is to create
123 # PEP 3147 pyc/pyo files.
124 retcode = subprocess.call(
125 (sys.executable, '-m', 'compileall', '-q', self.pkgdir))
126 self.assertEqual(retcode, 0)
127 # Verify the __pycache__ directory contents.
128 cachedir = os.path.join(self.pkgdir, '__pycache__')
129 self.assertTrue(os.path.exists(cachedir))
130 ext = ('pyc' if __debug__ else 'pyo')
131 expected = sorted(base.format(imp.get_tag(), ext) for base in
132 ('__init__.{}.{}', 'bar.{}.{}'))
133 self.assertEqual(sorted(os.listdir(cachedir)), expected)
134 # Make sure there are no .pyc files in the source directory.
135 self.assertFalse([pyc_file for pyc_file in os.listdir(self.pkgdir)
136 if pyc_file.endswith(ext)])
137
138 def test_legacy_paths(self):
139 # Ensure that with the proper switch, compileall leaves legacy
140 # pyc/pyo files, and no __pycache__ directory.
141 retcode = subprocess.call(
142 (sys.executable, '-m', 'compileall', '-b', '-q', self.pkgdir))
143 self.assertEqual(retcode, 0)
144 # Verify the __pycache__ directory contents.
145 cachedir = os.path.join(self.pkgdir, '__pycache__')
146 self.assertFalse(os.path.exists(cachedir))
147 ext = ('pyc' if __debug__ else 'pyo')
148 expected = [base.format(ext) for base in ('__init__.{}', 'bar.{}')]
149 expected.extend(['__init__.py', 'bar.py'])
150 expected.sort()
151 self.assertEqual(sorted(os.listdir(self.pkgdir)), expected)
152
153
98def test_main():154def test_main():
99 support.run_unittest(CompileallTests,155 support.run_unittest(
100 EncodingTest)156 CommandLineTests,
157 CompileallTests,
158 EncodingTest,
159 )
101160
102161
103if __name__ == "__main__":162if __name__ == "__main__":
104163
=== modified file 'Lib/test/test_frozen.py'
--- Lib/test/test_frozen.py 2010-03-14 10:45:50 +0000
+++ Lib/test/test_frozen.py 2010-04-16 18:55:41 +0000
@@ -11,7 +11,7 @@
11 except ImportError as x:11 except ImportError as x:
12 self.fail("import __hello__ failed:" + str(x))12 self.fail("import __hello__ failed:" + str(x))
13 self.assertEqual(__hello__.initialized, True)13 self.assertEqual(__hello__.initialized, True)
14 self.assertEqual(len(dir(__hello__)), 6, dir(__hello__))14 self.assertEqual(len(dir(__hello__)), 7, dir(__hello__))
1515
16 try:16 try:
17 import __phello__17 import __phello__
@@ -19,9 +19,9 @@
19 self.fail("import __phello__ failed:" + str(x))19 self.fail("import __phello__ failed:" + str(x))
20 self.assertEqual(__phello__.initialized, True)20 self.assertEqual(__phello__.initialized, True)
21 if not "__phello__.spam" in sys.modules:21 if not "__phello__.spam" in sys.modules:
22 self.assertEqual(len(dir(__phello__)), 7, dir(__phello__))
23 else:
24 self.assertEqual(len(dir(__phello__)), 8, dir(__phello__))22 self.assertEqual(len(dir(__phello__)), 8, dir(__phello__))
23 else:
24 self.assertEqual(len(dir(__phello__)), 9, dir(__phello__))
25 self.assertEquals(__phello__.__path__, [__phello__.__name__])25 self.assertEquals(__phello__.__path__, [__phello__.__name__])
2626
27 try:27 try:
@@ -29,8 +29,8 @@
29 except ImportError as x:29 except ImportError as x:
30 self.fail("import __phello__.spam failed:" + str(x))30 self.fail("import __phello__.spam failed:" + str(x))
31 self.assertEqual(__phello__.spam.initialized, True)31 self.assertEqual(__phello__.spam.initialized, True)
32 self.assertEqual(len(dir(__phello__.spam)), 6)32 self.assertEqual(len(dir(__phello__.spam)), 7)
33 self.assertEqual(len(dir(__phello__)), 8)33 self.assertEqual(len(dir(__phello__)), 9)
3434
35 try:35 try:
36 import __phello__.foo36 import __phello__.foo
3737
=== modified file 'Lib/test/test_imp.py'
--- Lib/test/test_imp.py 2010-03-20 20:31:34 +0000
+++ Lib/test/test_imp.py 2010-04-16 18:55:41 +0000
@@ -1,6 +1,7 @@
1import imp1import imp
2import os2import os
3import os.path3import os.path
4import shutil
4import sys5import sys
5import unittest6import unittest
6from test import support7from test import support
@@ -139,7 +140,8 @@
139 mod = imp.load_source(temp_mod_name, temp_mod_name + '.py')140 mod = imp.load_source(temp_mod_name, temp_mod_name + '.py')
140 self.assertEqual(mod.a, 1)141 self.assertEqual(mod.a, 1)
141142
142 mod = imp.load_compiled(temp_mod_name, temp_mod_name + '.pyc')143 mod = imp.load_compiled(
144 temp_mod_name, imp.cache_from_source(temp_mod_name + '.py'))
143 self.assertEqual(mod.a, 1)145 self.assertEqual(mod.a, 1)
144146
145 if not os.path.exists(test_package_name):147 if not os.path.exists(test_package_name):
@@ -184,11 +186,132 @@
184 imp.reload(marshal)186 imp.reload(marshal)
185187
186188
189class PEP3147Tests(unittest.TestCase):
190 """Tests of PEP 3147."""
191
192 tag = imp.get_tag()
193
194 def test_cache_from_source(self):
195 # Given the path to a .py file, return the path to its PEP 3147
196 # defined .pyc file (i.e. under __pycache__).
197 self.assertEqual(
198 imp.cache_from_source('/foo/bar/baz/qux.py', True),
199 '/foo/bar/baz/__pycache__/qux.{}.pyc'.format(self.tag))
200
201 def test_cache_from_source_optimized(self):
202 # Given the path to a .py file, return the path to its PEP 3147
203 # defined .pyo file (i.e. under __pycache__).
204 self.assertEqual(
205 imp.cache_from_source('/foo/bar/baz/qux.py', False),
206 '/foo/bar/baz/__pycache__/qux.{}.pyo'.format(self.tag))
207
208 def test_cache_from_source_cwd(self):
209 self.assertEqual(imp.cache_from_source('foo.py', True),
210 os.sep.join(('__pycache__',
211 'foo.{}.pyc'.format(self.tag))))
212
213 def test_cache_from_source_override(self):
214 # When debug_override is not None, it can be any true-ish or false-ish
215 # value.
216 self.assertEqual(
217 imp.cache_from_source('/foo/bar/baz.py', []),
218 '/foo/bar/__pycache__/baz.{}.pyo'.format(self.tag))
219 self.assertEqual(
220 imp.cache_from_source('/foo/bar/baz.py', [17]),
221 '/foo/bar/__pycache__/baz.{}.pyc'.format(self.tag))
222 # However if the bool-ishness can't be determined, the exception
223 # propagates.
224 class Bearish:
225 def __bool__(self): raise RuntimeError
226 self.assertRaises(
227 RuntimeError,
228 imp.cache_from_source, '/foo/bar/baz.py', Bearish())
229
230 @unittest.skipIf(os.altsep is None,
231 'test meaningful only where os.altsep is defined')
232 def test_altsep_cache_from_source(self):
233 # Windows path and PEP 3147.
234 self.assertEqual(
235 imp.cache_from_source('\\foo\\bar\\baz\\qux.py', True),
236 '\\foo\\bar\\baz\\__pycache__\\qux.{}.pyc'.format(self.tag))
237
238 @unittest.skipIf(os.altsep is None,
239 'test meaningful only where os.altsep is defined')
240 def test_altsep_and_sep_cache_from_source(self):
241 # Windows path and PEP 3147 where altsep is right of sep.
242 self.assertEqual(
243 imp.cache_from_source('\\foo\\bar/baz\\qux.py', True),
244 '\\foo\\bar/baz\\__pycache__\\qux.{}.pyc'.format(self.tag))
245
246 @unittest.skipIf(os.altsep is None,
247 'test meaningful only where os.altsep is defined')
248 def test_sep_altsep_and_sep_cache_from_source(self):
249 # Windows path and PEP 3147 where sep is right of altsep.
250 self.assertEqual(
251 imp.cache_from_source('\\foo\\bar\\baz/qux.py', True),
252 '\\foo\\bar\\baz/__pycache__/qux.{}.pyc'.format(self.tag))
253
254 def test_source_from_cache(self):
255 # Given the path to a PEP 3147 defined .pyc file, return the path to
256 # its source. This tests the good path.
257 self.assertEqual(imp.source_from_cache(
258 '/foo/bar/baz/__pycache__/qux.{}.pyc'.format(self.tag)),
259 '/foo/bar/baz/qux.py')
260
261 def test_source_from_cache_bad_path(self):
262 # When the path to a pyc file is not in PEP 3147 format, a ValueError
263 # is raised.
264 self.assertRaises(
265 ValueError, imp.source_from_cache, '/foo/bar/bazqux.pyc')
266
267 def test_source_from_cache_no_slash(self):
268 # No slashes at all in path -> ValueError
269 self.assertRaises(
270 ValueError, imp.source_from_cache, 'foo.cpython-32.pyc')
271
272 def test_source_from_cache_too_few_dots(self):
273 # Too few dots in final path component -> ValueError
274 self.assertRaises(
275 ValueError, imp.source_from_cache, '__pycache__/foo.pyc')
276
277 def test_source_from_cache_too_many_dots(self):
278 # Too many dots in final path component -> ValueError
279 self.assertRaises(
280 ValueError, imp.source_from_cache,
281 '__pycache__/foo.cpython-32.foo.pyc')
282
283 def test_source_from_cache_no__pycache__(self):
284 # Another problem with the path -> ValueError
285 self.assertRaises(
286 ValueError, imp.source_from_cache,
287 '/foo/bar/foo.cpython-32.foo.pyc')
288
289 def test_package___file__(self):
290 # Test that a package's __file__ points to the right source directory.
291 os.mkdir('pep3147')
292 sys.path.insert(0, os.curdir)
293 def cleanup():
294 if sys.path[0] == os.curdir:
295 del sys.path[0]
296 shutil.rmtree('pep3147')
297 self.addCleanup(cleanup)
298 # Touch the __init__.py file.
299 with open('pep3147/__init__.py', 'w'):
300 pass
301 m = __import__('pep3147')
302 # Ensure we load the pyc file.
303 support.forget('pep3147')
304 m = __import__('pep3147')
305 self.assertEqual(m.__file__,
306 os.sep.join(('.', 'pep3147', '__init__.py')))
307
308
187def test_main():309def test_main():
188 tests = [310 tests = [
189 ImportTests,311 ImportTests,
312 PEP3147Tests,
190 ReloadTests,313 ReloadTests,
191 ]314 ]
192 try:315 try:
193 import _thread316 import _thread
194 except ImportError:317 except ImportError:
195318
=== modified file 'Lib/test/test_import.py'
--- Lib/test/test_import.py 2010-03-17 20:29:51 +0000
+++ Lib/test/test_import.py 2010-04-16 18:55:41 +0000
@@ -1,4 +1,5 @@
1import builtins1import builtins
2import errno
2import imp3import imp
3import marshal4import marshal
4import os5import os
@@ -8,8 +9,11 @@
8import stat9import stat
9import sys10import sys
10import unittest11import unittest
11from test.support import (unlink, TESTFN, unload, run_unittest, is_jython,12
12 check_warnings, EnvironmentVarGuard, swap_attr, swap_item)13from test.support import (
14 EnvironmentVarGuard, TESTFN, check_warnings, forget, is_jython,
15 make_legacy_pyc, rmtree, run_unittest, swap_attr, swap_item, temp_umask,
16 unlink, unload)
1317
1418
15def remove_files(name):19def remove_files(name):
@@ -19,12 +23,18 @@
19 name + ".pyw",23 name + ".pyw",
20 name + "$py.class"):24 name + "$py.class"):
21 unlink(f)25 unlink(f)
26 try:
27 shutil.rmtree('__pycache__')
28 except OSError as error:
29 if error.errno != errno.ENOENT:
30 raise
2231
2332
24class ImportTests(unittest.TestCase):33class ImportTests(unittest.TestCase):
2534
26 def tearDown(self):35 def tearDown(self):
27 unload(TESTFN)36 unload(TESTFN)
37
28 setUp = tearDown38 setUp = tearDown
2939
30 def test_case_sensitivity(self):40 def test_case_sensitivity(self):
@@ -53,8 +63,8 @@
53 pyc = TESTFN + ".pyc"63 pyc = TESTFN + ".pyc"
5464
55 with open(source, "w") as f:65 with open(source, "w") as f:
56 print("# This tests Python's ability to import a", ext, "file.",66 print("# This tests Python's ability to import a",
57 file=f)67 ext, "file.", file=f)
58 a = random.randrange(1000)68 a = random.randrange(1000)
59 b = random.randrange(1000)69 b = random.randrange(1000)
60 print("a =", a, file=f)70 print("a =", a, file=f)
@@ -73,10 +83,10 @@
73 self.assertEqual(mod.b, b,83 self.assertEqual(mod.b, b,
74 "module loaded (%s) but contents invalid" % mod)84 "module loaded (%s) but contents invalid" % mod)
75 finally:85 finally:
86 forget(TESTFN)
76 unlink(source)87 unlink(source)
77 unlink(pyc)88 unlink(pyc)
78 unlink(pyo)89 unlink(pyo)
79 unload(TESTFN)
8090
81 sys.path.insert(0, os.curdir)91 sys.path.insert(0, os.curdir)
82 try:92 try:
@@ -87,32 +97,31 @@
87 finally:97 finally:
88 del sys.path[0]98 del sys.path[0]
8999
90 @unittest.skipUnless(os.name == 'posix', "test meaningful only on posix systems")100 @unittest.skipUnless(os.name == 'posix',
101 "test meaningful only on posix systems")
91 def test_execute_bit_not_copied(self):102 def test_execute_bit_not_copied(self):
92 # Issue 6070: under posix .pyc files got their execute bit set if103 # Issue 6070: under posix .pyc files got their execute bit set if
93 # the .py file had the execute bit set, but they aren't executable.104 # the .py file had the execute bit set, but they aren't executable.
94 oldmask = os.umask(0o022)105 with temp_umask(0o022):
95 sys.path.insert(0, os.curdir)106 sys.path.insert(0, os.curdir)
96 try:107 try:
97 fname = TESTFN + os.extsep + "py"108 fname = TESTFN + os.extsep + "py"
98 f = open(fname, 'w').close()109 f = open(fname, 'w').close()
99 os.chmod(fname, (stat.S_IRUSR | stat.S_IRGRP | stat.S_IROTH |110 os.chmod(fname, (stat.S_IRUSR | stat.S_IRGRP | stat.S_IROTH |
100 stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH))111 stat.S_IXUSR | stat.S_IXGRP | stat.S_IXOTH))
101 __import__(TESTFN)112 __import__(TESTFN)
102 fn = fname + 'c'113 fn = imp.cache_from_source(fname)
103 if not os.path.exists(fn):
104 fn = fname + 'o'
105 if not os.path.exists(fn):114 if not os.path.exists(fn):
106 self.fail("__import__ did not result in creation of "115 self.fail("__import__ did not result in creation of "
107 "either a .pyc or .pyo file")116 "either a .pyc or .pyo file")
108 s = os.stat(fn)117 s = os.stat(fn)
109 self.assertEqual(stat.S_IMODE(s.st_mode),118 self.assertEqual(
110 stat.S_IRUSR | stat.S_IRGRP | stat.S_IROTH)119 stat.S_IMODE(s.st_mode),
111 finally:120 stat.S_IRUSR | stat.S_IRGRP | stat.S_IROTH)
112 os.umask(oldmask)121 finally:
113 remove_files(TESTFN)122 del sys.path[0]
114 unload(TESTFN)123 remove_files(TESTFN)
115 del sys.path[0]124 unload(TESTFN)
116125
117 def test_imp_module(self):126 def test_imp_module(self):
118 # Verify that the imp module can correctly load and find .py files127 # Verify that the imp module can correctly load and find .py files
@@ -144,10 +153,12 @@
144 f.write('"",\n')153 f.write('"",\n')
145 f.write(']')154 f.write(']')
146155
147 # Compile & remove .py file, we only need .pyc (or .pyo).156 # Compile & remove .py file, we only need .pyc (or .pyo), but that
157 # must be relocated to the PEP 3147 bytecode-only location.
148 with open(filename, 'r') as f:158 with open(filename, 'r') as f:
149 py_compile.compile(filename)159 py_compile.compile(filename)
150 unlink(filename)160 unlink(filename)
161 make_legacy_pyc(filename)
151162
152 # Need to be able to load from current dir.163 # Need to be able to load from current dir.
153 sys.path.append('')164 sys.path.append('')
@@ -247,8 +258,9 @@
247 self.assertTrue(mod.__file__.endswith('.py'))258 self.assertTrue(mod.__file__.endswith('.py'))
248 os.remove(source)259 os.remove(source)
249 del sys.modules[TESTFN]260 del sys.modules[TESTFN]
261 make_legacy_pyc(source)
250 mod = __import__(TESTFN)262 mod = __import__(TESTFN)
251 ext = mod.__file__[-4:]263 base, ext = os.path.splitext(mod.__file__)
252 self.assertIn(ext, ('.pyc', '.pyo'))264 self.assertIn(ext, ('.pyc', '.pyo'))
253 finally:265 finally:
254 del sys.path[0]266 del sys.path[0]
@@ -298,7 +310,7 @@
298"""310"""
299 dir_name = os.path.abspath(TESTFN)311 dir_name = os.path.abspath(TESTFN)
300 file_name = os.path.join(dir_name, module_name) + os.extsep + "py"312 file_name = os.path.join(dir_name, module_name) + os.extsep + "py"
301 compiled_name = file_name + ("c" if __debug__ else "o")313 compiled_name = imp.cache_from_source(file_name)
302314
303 def setUp(self):315 def setUp(self):
304 self.sys_path = sys.path[:]316 self.sys_path = sys.path[:]
@@ -346,8 +358,9 @@
346 target = "another_module.py"358 target = "another_module.py"
347 py_compile.compile(self.file_name, dfile=target)359 py_compile.compile(self.file_name, dfile=target)
348 os.remove(self.file_name)360 os.remove(self.file_name)
361 pyc_file = make_legacy_pyc(self.file_name)
349 mod = self.import_module()362 mod = self.import_module()
350 self.assertEqual(mod.module_filename, self.compiled_name)363 self.assertEqual(mod.module_filename, pyc_file)
351 self.assertEqual(mod.code_filename, target)364 self.assertEqual(mod.code_filename, target)
352 self.assertEqual(mod.func_filename, target)365 self.assertEqual(mod.func_filename, target)
353366
@@ -476,10 +489,143 @@
476 self.assertEqual(foo(), os)489 self.assertEqual(foo(), os)
477490
478491
492class PycacheTests(unittest.TestCase):
493 # Test the various PEP 3147 related behaviors.
494
495 tag = imp.get_tag()
496
497 def _clean(self):
498 forget(TESTFN)
499 rmtree('__pycache__')
500 unlink(self.source)
501
502 def setUp(self):
503 self.source = TESTFN + '.py'
504 self._clean()
505 with open(self.source, 'w') as fp:
506 print('# This is a test file written by test_import.py', file=fp)
507 sys.path.insert(0, os.curdir)
508
509 def tearDown(self):
510 assert sys.path[0] == os.curdir, 'Unexpected sys.path[0]'
511 del sys.path[0]
512 self._clean()
513
514 def test_import_pyc_path(self):
515 self.assertFalse(os.path.exists('__pycache__'))
516 __import__(TESTFN)
517 self.assertTrue(os.path.exists('__pycache__'))
518 self.assertTrue(os.path.exists(os.path.join(
519 '__pycache__', '{}.{}.pyc'.format(TESTFN, self.tag))))
520
521 @unittest.skipUnless(os.name == 'posix',
522 "test meaningful only on posix systems")
523 def test_unwritable_directory(self):
524 # When the umask causes the new __pycache__ directory to be
525 # unwritable, the import still succeeds but no .pyc file is written.
526 with temp_umask(0o222):
527 __import__(TESTFN)
528 self.assertTrue(os.path.exists('__pycache__'))
529 self.assertFalse(os.path.exists(os.path.join(
530 '__pycache__', '{}.{}.pyc'.format(TESTFN, self.tag))))
531
532 def test_missing_source(self):
533 # With PEP 3147 cache layout, removing the source but leaving the pyc
534 # file does not satisfy the import.
535 __import__(TESTFN)
536 pyc_file = imp.cache_from_source(self.source)
537 self.assertTrue(os.path.exists(pyc_file))
538 os.remove(self.source)
539 forget(TESTFN)
540 self.assertRaises(ImportError, __import__, TESTFN)
541
542 def test_missing_source_legacy(self):
543 # Like test_missing_source() except that for backward compatibility,
544 # when the pyc file lives where the py file would have been (and named
545 # without the tag), it is importable. The __file__ of the imported
546 # module is the pyc location.
547 __import__(TESTFN)
548 # pyc_file gets removed in _clean() via tearDown().
549 pyc_file = make_legacy_pyc(self.source)
550 os.remove(self.source)
551 unload(TESTFN)
552 m = __import__(TESTFN)
553 self.assertEqual(m.__file__,
554 os.path.join(os.curdir, os.path.relpath(pyc_file)))
555
556 def test___cached__(self):
557 # Modules now also have an __cached__ that points to the pyc file.
558 m = __import__(TESTFN)
559 pyc_file = imp.cache_from_source(TESTFN + '.py')
560 self.assertEqual(m.__cached__, os.path.join(os.curdir, pyc_file))
561
562 def test___cached___legacy_pyc(self):
563 # Like test___cached__() except that for backward compatibility,
564 # when the pyc file lives where the py file would have been (and named
565 # without the tag), it is importable. The __cached__ of the imported
566 # module is the pyc location.
567 __import__(TESTFN)
568 # pyc_file gets removed in _clean() via tearDown().
569 pyc_file = make_legacy_pyc(self.source)
570 os.remove(self.source)
571 unload(TESTFN)
572 m = __import__(TESTFN)
573 self.assertEqual(m.__cached__,
574 os.path.join(os.curdir, os.path.relpath(pyc_file)))
575
576 def test_package___cached__(self):
577 # Like test___cached__ but for packages.
578 def cleanup():
579 shutil.rmtree('pep3147')
580 os.mkdir('pep3147')
581 self.addCleanup(cleanup)
582 # Touch the __init__.py
583 with open(os.path.join('pep3147', '__init__.py'), 'w'):
584 pass
585 with open(os.path.join('pep3147', 'foo.py'), 'w'):
586 pass
587 unload('pep3147.foo')
588 unload('pep3147')
589 m = __import__('pep3147.foo')
590 init_pyc = imp.cache_from_source(
591 os.path.join('pep3147', '__init__.py'))
592 self.assertEqual(m.__cached__, os.path.join(os.curdir, init_pyc))
593 foo_pyc = imp.cache_from_source(os.path.join('pep3147', 'foo.py'))
594 self.assertEqual(sys.modules['pep3147.foo'].__cached__,
595 os.path.join(os.curdir, foo_pyc))
596
597 def test_package___cached___from_pyc(self):
598 # Like test___cached__ but ensuring __cached__ when imported from a
599 # PEP 3147 pyc file.
600 def cleanup():
601 shutil.rmtree('pep3147')
602 os.mkdir('pep3147')
603 self.addCleanup(cleanup)
604 unload('pep3147.foo')
605 unload('pep3147')
606 # Touch the __init__.py
607 with open(os.path.join('pep3147', '__init__.py'), 'w'):
608 pass
609 with open(os.path.join('pep3147', 'foo.py'), 'w'):
610 pass
611 m = __import__('pep3147.foo')
612 unload('pep3147.foo')
613 unload('pep3147')
614 m = __import__('pep3147.foo')
615 init_pyc = imp.cache_from_source(
616 os.path.join('pep3147', '__init__.py'))
617 self.assertEqual(m.__cached__, os.path.join(os.curdir, init_pyc))
618 foo_pyc = imp.cache_from_source(os.path.join('pep3147', 'foo.py'))
619 self.assertEqual(sys.modules['pep3147.foo'].__cached__,
620 os.path.join(os.curdir, foo_pyc))
621
622
479def test_main(verbose=None):623def test_main(verbose=None):
480 run_unittest(ImportTests, PycRewritingTests, PathsTests, RelativeImportTests,624 run_unittest(ImportTests, PycacheTests,
625 PycRewritingTests, PathsTests, RelativeImportTests,
481 OverridingImportBuiltinTests)626 OverridingImportBuiltinTests)
482627
628
483if __name__ == '__main__':629if __name__ == '__main__':
484 # Test needs to be a package, so we can do relative imports.630 # Test needs to be a package, so we can do relative imports.
485 from test.test_import import test_main631 from test.test_import import test_main
486632
=== modified file 'Lib/test/test_pkg.py'
--- Lib/test/test_pkg.py 2010-02-24 01:46:21 +0000
+++ Lib/test/test_pkg.py 2010-04-16 18:55:41 +0000
@@ -196,14 +196,14 @@
196196
197 import t5197 import t5
198 self.assertEqual(fixdir(dir(t5)),198 self.assertEqual(fixdir(dir(t5)),
199 ['__doc__', '__file__', '__name__',199 ['__cached__', '__doc__', '__file__', '__name__',
200 '__package__', '__path__', 'foo', 'string', 't5'])200 '__package__', '__path__', 'foo', 'string', 't5'])
201 self.assertEqual(fixdir(dir(t5.foo)),201 self.assertEqual(fixdir(dir(t5.foo)),
202 ['__doc__', '__file__', '__name__', '__package__',202 ['__cached__', '__doc__', '__file__', '__name__',
203 'string'])203 '__package__', 'string'])
204 self.assertEqual(fixdir(dir(t5.string)),204 self.assertEqual(fixdir(dir(t5.string)),
205 ['__doc__', '__file__', '__name__','__package__',205 ['__cached__', '__doc__', '__file__', '__name__',
206 'spam'])206 '__package__', 'spam'])
207207
208 def test_6(self):208 def test_6(self):
209 hier = [209 hier = [
@@ -218,13 +218,13 @@
218218
219 import t6219 import t6
220 self.assertEqual(fixdir(dir(t6)),220 self.assertEqual(fixdir(dir(t6)),
221 ['__all__', '__doc__', '__file__',221 ['__all__', '__cached__', '__doc__', '__file__',
222 '__name__', '__package__', '__path__'])222 '__name__', '__package__', '__path__'])
223 s = """223 s = """
224 import t6224 import t6
225 from t6 import *225 from t6 import *
226 self.assertEqual(fixdir(dir(t6)),226 self.assertEqual(fixdir(dir(t6)),
227 ['__all__', '__doc__', '__file__',227 ['__all__', '__cached__', '__doc__', '__file__',
228 '__name__', '__package__', '__path__',228 '__name__', '__package__', '__path__',
229 'eggs', 'ham', 'spam'])229 'eggs', 'ham', 'spam'])
230 self.assertEqual(dir(), ['eggs', 'ham', 'self', 'spam', 't6'])230 self.assertEqual(dir(), ['eggs', 'ham', 'self', 'spam', 't6'])
@@ -252,18 +252,18 @@
252 t7, sub, subsub = None, None, None252 t7, sub, subsub = None, None, None
253 import t7 as tas253 import t7 as tas
254 self.assertEqual(fixdir(dir(tas)),254 self.assertEqual(fixdir(dir(tas)),
255 ['__doc__', '__file__', '__name__',255 ['__cached__', '__doc__', '__file__', '__name__',
256 '__package__', '__path__'])256 '__package__', '__path__'])
257 self.assertFalse(t7)257 self.assertFalse(t7)
258 from t7 import sub as subpar258 from t7 import sub as subpar
259 self.assertEqual(fixdir(dir(subpar)),259 self.assertEqual(fixdir(dir(subpar)),
260 ['__doc__', '__file__', '__name__',260 ['__cached__', '__doc__', '__file__', '__name__',
261 '__package__', '__path__'])261 '__package__', '__path__'])
262 self.assertFalse(t7)262 self.assertFalse(t7)
263 self.assertFalse(sub)263 self.assertFalse(sub)
264 from t7.sub import subsub as subsubsub264 from t7.sub import subsub as subsubsub
265 self.assertEqual(fixdir(dir(subsubsub)),265 self.assertEqual(fixdir(dir(subsubsub)),
266 ['__doc__', '__file__', '__name__',266 ['__cached__', '__doc__', '__file__', '__name__',
267 '__package__', '__path__', 'spam'])267 '__package__', '__path__', 'spam'])
268 self.assertFalse(t7)268 self.assertFalse(t7)
269 self.assertFalse(sub)269 self.assertFalse(sub)
270270
=== modified file 'Lib/test/test_pkgimport.py'
--- Lib/test/test_pkgimport.py 2010-01-23 15:40:09 +0000
+++ Lib/test/test_pkgimport.py 2010-04-16 18:55:41 +0000
@@ -1,5 +1,12 @@
1import os, sys, string, random, tempfile, unittest1import os
2import sys
3import shutil
4import string
5import random
6import tempfile
7import unittest
28
9from imp import cache_from_source
3from test.support import run_unittest10from test.support import run_unittest
411
5class TestImport(unittest.TestCase):12class TestImport(unittest.TestCase):
@@ -26,22 +33,17 @@
26 self.module_path = os.path.join(self.package_dir, 'foo.py')33 self.module_path = os.path.join(self.package_dir, 'foo.py')
2734
28 def tearDown(self):35 def tearDown(self):
29 for file in os.listdir(self.package_dir):36 shutil.rmtree(self.test_dir)
30 os.remove(os.path.join(self.package_dir, file))
31 os.rmdir(self.package_dir)
32 os.rmdir(self.test_dir)
33 self.assertNotEqual(sys.path.count(self.test_dir), 0)37 self.assertNotEqual(sys.path.count(self.test_dir), 0)
34 sys.path.remove(self.test_dir)38 sys.path.remove(self.test_dir)
35 self.remove_modules()39 self.remove_modules()
3640
37 def rewrite_file(self, contents):41 def rewrite_file(self, contents):
38 for extension in "co":42 compiled_path = cache_from_source(self.module_path)
39 compiled_path = self.module_path + extension43 if os.path.exists(compiled_path):
40 if os.path.exists(compiled_path):44 os.remove(compiled_path)
41 os.remove(compiled_path)45 with open(self.module_path, 'w') as f:
42 f = open(self.module_path, 'w')46 f.write(contents)
43 f.write(contents)
44 f.close()
4547
46 def test_package_import__semantics(self):48 def test_package_import__semantics(self):
4749
4850
=== modified file 'Lib/test/test_pydoc.py'
--- Lib/test/test_pydoc.py 2010-03-31 02:19:28 +0000
+++ Lib/test/test_pydoc.py 2010-04-16 18:55:41 +0000
@@ -19,8 +19,7 @@
19if hasattr(pydoc_mod, "__loader__"):19if hasattr(pydoc_mod, "__loader__"):
20 del pydoc_mod.__loader__20 del pydoc_mod.__loader__
2121
22expected_text_pattern = \22expected_text_pattern = """
23"""
24NAME23NAME
25 test.pydoc_mod - This is a test module for test_pydoc24 test.pydoc_mod - This is a test module for test_pydoc
2625
@@ -87,8 +86,7 @@
87 Nobody86 Nobody
88""".strip()87""".strip()
8988
90expected_html_pattern = \89expected_html_pattern = """
91"""
92<table width="100%%" cellspacing=0 cellpadding=2 border=0 summary="heading">90<table width="100%%" cellspacing=0 cellpadding=2 border=0 summary="heading">
93<tr bgcolor="#7799ee">91<tr bgcolor="#7799ee">
94<td valign=bottom>&nbsp;<br>92<td valign=bottom>&nbsp;<br>
@@ -186,7 +184,7 @@
186\x20\x20\x20\x20184\x20\x20\x20\x20
187<tr><td bgcolor="#7799ee"><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</tt></td><td>&nbsp;</td>185<tr><td bgcolor="#7799ee"><tt>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;</tt></td><td>&nbsp;</td>
188<td width="100%%">Nobody</td></tr></table>186<td width="100%%">Nobody</td></tr></table>
189""".strip()187""".strip() # ' <- emacs turd
190188
191189
192# output pattern for missing module190# output pattern for missing module
@@ -287,7 +285,8 @@
287 ('i_am_not_here', 'i_am_not_here'),285 ('i_am_not_here', 'i_am_not_here'),
288 ('test.i_am_not_here_either', 'i_am_not_here_either'),286 ('test.i_am_not_here_either', 'i_am_not_here_either'),
289 ('test.i_am_not_here.neither_am_i', 'i_am_not_here.neither_am_i'),287 ('test.i_am_not_here.neither_am_i', 'i_am_not_here.neither_am_i'),
290 ('i_am_not_here.{}'.format(modname), 'i_am_not_here.{}'.format(modname)),288 ('i_am_not_here.{}'.format(modname),
289 'i_am_not_here.{}'.format(modname)),
291 ('test.{}'.format(modname), modname),290 ('test.{}'.format(modname), modname),
292 )291 )
293292
@@ -304,9 +303,8 @@
304 fullmodname = os.path.join(TESTFN, modname)303 fullmodname = os.path.join(TESTFN, modname)
305 sourcefn = fullmodname + os.extsep + "py"304 sourcefn = fullmodname + os.extsep + "py"
306 for importstring, expectedinmsg in testpairs:305 for importstring, expectedinmsg in testpairs:
307 f = open(sourcefn, 'w')306 with open(sourcefn, 'w') as f:
308 f.write("import {}\n".format(importstring))307 f.write("import {}\n".format(importstring))
309 f.close()
310 try:308 try:
311 result = run_pydoc(modname).decode("ascii")309 result = run_pydoc(modname).decode("ascii")
312 finally:310 finally:
313311
=== modified file 'Lib/test/test_runpy.py'
--- Lib/test/test_runpy.py 2010-01-19 00:09:57 +0000
+++ Lib/test/test_runpy.py 2010-04-16 18:55:41 +0000
@@ -5,9 +5,10 @@
5import sys5import sys
6import re6import re
7import tempfile7import tempfile
8from test.support import verbose, run_unittest, forget8import py_compile
9from test.script_helper import (temp_dir, make_script, compile_script,9from test.support import forget, make_legacy_pyc, run_unittest, verbose
10 make_pkg, make_zip_script, make_zip_pkg)10from test.script_helper import (
11 make_pkg, make_script, make_zip_pkg, make_zip_script, temp_dir)
1112
1213
13from runpy import _run_code, _run_module_code, run_module, run_path14from runpy import _run_code, _run_module_code, run_module, run_path
@@ -45,6 +46,7 @@
45 self.assertEqual(d["result"], self.expected_result)46 self.assertEqual(d["result"], self.expected_result)
46 self.assertIs(d["__name__"], None)47 self.assertIs(d["__name__"], None)
47 self.assertIs(d["__file__"], None)48 self.assertIs(d["__file__"], None)
49 self.assertIs(d["__cached__"], None)
48 self.assertIs(d["__loader__"], None)50 self.assertIs(d["__loader__"], None)
49 self.assertIs(d["__package__"], None)51 self.assertIs(d["__package__"], None)
50 self.assertIs(d["run_argv0"], saved_argv0)52 self.assertIs(d["run_argv0"], saved_argv0)
@@ -73,6 +75,7 @@
73 self.assertTrue(d2["run_name_in_sys_modules"])75 self.assertTrue(d2["run_name_in_sys_modules"])
74 self.assertTrue(d2["module_in_sys_modules"])76 self.assertTrue(d2["module_in_sys_modules"])
75 self.assertIs(d2["__file__"], file)77 self.assertIs(d2["__file__"], file)
78 self.assertIs(d2["__cached__"], None)
76 self.assertIs(d2["run_argv0"], file)79 self.assertIs(d2["run_argv0"], file)
77 self.assertIs(d2["__loader__"], loader)80 self.assertIs(d2["__loader__"], loader)
78 self.assertIs(d2["__package__"], package)81 self.assertIs(d2["__package__"], package)
@@ -170,6 +173,7 @@
170 del d1 # Ensure __loader__ entry doesn't keep file open173 del d1 # Ensure __loader__ entry doesn't keep file open
171 __import__(mod_name)174 __import__(mod_name)
172 os.remove(mod_fname)175 os.remove(mod_fname)
176 make_legacy_pyc(mod_fname)
173 if verbose: print("Running from compiled:", mod_name)177 if verbose: print("Running from compiled:", mod_name)
174 d2 = run_module(mod_name) # Read from bytecode178 d2 = run_module(mod_name) # Read from bytecode
175 self.assertIn("x", d2)179 self.assertIn("x", d2)
@@ -192,6 +196,7 @@
192 del d1 # Ensure __loader__ entry doesn't keep file open196 del d1 # Ensure __loader__ entry doesn't keep file open
193 __import__(mod_name)197 __import__(mod_name)
194 os.remove(mod_fname)198 os.remove(mod_fname)
199 make_legacy_pyc(mod_fname)
195 if verbose: print("Running from compiled:", pkg_name)200 if verbose: print("Running from compiled:", pkg_name)
196 d2 = run_module(pkg_name) # Read from bytecode201 d2 = run_module(pkg_name) # Read from bytecode
197 self.assertIn("x", d2)202 self.assertIn("x", d2)
@@ -246,6 +251,7 @@
246 del d1 # Ensure __loader__ entry doesn't keep file open251 del d1 # Ensure __loader__ entry doesn't keep file open
247 __import__(mod_name)252 __import__(mod_name)
248 os.remove(mod_fname)253 os.remove(mod_fname)
254 make_legacy_pyc(mod_fname)
249 if verbose: print("Running from compiled:", mod_name)255 if verbose: print("Running from compiled:", mod_name)
250 d2 = run_module(mod_name, run_name=run_name) # Read from bytecode256 d2 = run_module(mod_name, run_name=run_name) # Read from bytecode
251 self.assertIn("__package__", d2)257 self.assertIn("__package__", d2)
@@ -313,6 +319,7 @@
313 result = run_path(script_name)319 result = run_path(script_name)
314 self.assertEqual(result["__name__"], expected_name)320 self.assertEqual(result["__name__"], expected_name)
315 self.assertEqual(result["__file__"], expected_file)321 self.assertEqual(result["__file__"], expected_file)
322 self.assertEqual(result["__cached__"], None)
316 self.assertIn("argv0", result)323 self.assertIn("argv0", result)
317 self.assertEqual(result["argv0"], expected_argv0)324 self.assertEqual(result["argv0"], expected_argv0)
318 self.assertEqual(result["__package__"], expected_package)325 self.assertEqual(result["__package__"], expected_package)
@@ -332,7 +339,7 @@
332 with temp_dir() as script_dir:339 with temp_dir() as script_dir:
333 mod_name = 'script'340 mod_name = 'script'
334 script_name = self._make_test_script(script_dir, mod_name)341 script_name = self._make_test_script(script_dir, mod_name)
335 compiled_name = compile_script(script_name)342 compiled_name = py_compile.compile(script_name, doraise=True)
336 os.remove(script_name)343 os.remove(script_name)
337 self._check_script(compiled_name, "<run_path>", compiled_name,344 self._check_script(compiled_name, "<run_path>", compiled_name,
338 compiled_name, None)345 compiled_name, None)
@@ -348,9 +355,10 @@
348 with temp_dir() as script_dir:355 with temp_dir() as script_dir:
349 mod_name = '__main__'356 mod_name = '__main__'
350 script_name = self._make_test_script(script_dir, mod_name)357 script_name = self._make_test_script(script_dir, mod_name)
351 compiled_name = compile_script(script_name)358 compiled_name = py_compile.compile(script_name, doraise=True)
352 os.remove(script_name)359 os.remove(script_name)
353 self._check_script(script_dir, "<run_path>", compiled_name,360 legacy_pyc = make_legacy_pyc(script_name)
361 self._check_script(script_dir, "<run_path>", legacy_pyc,
354 script_dir, '')362 script_dir, '')
355363
356 def test_directory_error(self):364 def test_directory_error(self):
@@ -371,8 +379,9 @@
371 with temp_dir() as script_dir:379 with temp_dir() as script_dir:
372 mod_name = '__main__'380 mod_name = '__main__'
373 script_name = self._make_test_script(script_dir, mod_name)381 script_name = self._make_test_script(script_dir, mod_name)
374 compiled_name = compile_script(script_name)382 compiled_name = py_compile.compile(script_name, doraise=True)
375 zip_name, fname = make_zip_script(script_dir, 'test_zip', compiled_name)383 zip_name, fname = make_zip_script(script_dir, 'test_zip',
384 compiled_name)
376 self._check_script(zip_name, "<run_path>", fname, zip_name, '')385 self._check_script(zip_name, "<run_path>", fname, zip_name, '')
377386
378 def test_zipfile_error(self):387 def test_zipfile_error(self):
379388
=== modified file 'Lib/test/test_site.py'
--- Lib/test/test_site.py 2010-01-29 11:41:03 +0000
+++ Lib/test/test_site.py 2010-04-16 18:55:41 +0000
@@ -258,19 +258,38 @@
258 """Restore sys.path"""258 """Restore sys.path"""
259 sys.path[:] = self.sys_path259 sys.path[:] = self.sys_path
260260
261 def test_abs__file__(self):261 def test_abs_paths(self):
262 # Make sure all imported modules have their __file__ attribute262 # Make sure all imported modules have their __file__ and __cached__
263 # as an absolute path.263 # attributes as absolute paths. Arranging to put the Lib directory on
264 # Handled by abs__file__()264 # PYTHONPATH would cause the os module to have a relative path for
265 site.abs__file__()265 # __file__ if abs_paths() does not get run. sys and builtins (the
266 for module in (sys, os, builtins):266 # only other modules imported before site.py runs) do not have
267 try:267 # __file__ or __cached__ because they are built-in.
268 self.assertTrue(os.path.isabs(module.__file__), repr(module))268 parent = os.path.relpath(os.path.dirname(os.__file__))
269 except AttributeError:269 env = os.environ.copy()
270 continue270 env['PYTHONPATH'] = parent
271 # We could try everything in sys.modules; however, when regrtest.py271 command = 'import os; print(os.__file__, os.__cached__)'
272 # runs something like test_frozen before test_site, then we will272 # First, prove that with -S (no 'import site'), the paths are
273 # be testing things loaded *after* test_site did path normalization273 # relative.
274 proc = subprocess.Popen([sys.executable, '-S', '-c', command],
275 env=env,
276 stdout=subprocess.PIPE,
277 stderr=subprocess.PIPE)
278 stdout, stderr = proc.communicate()
279 self.assertEqual(proc.returncode, 0)
280 os__file__, os__cached__ = stdout.split()
281 self.assertFalse(os.path.isabs(os__file__))
282 self.assertFalse(os.path.isabs(os__cached__))
283 # Now, with 'import site', it works.
284 proc = subprocess.Popen([sys.executable, '-c', command],
285 env=env,
286 stdout=subprocess.PIPE,
287 stderr=subprocess.PIPE)
288 stdout, stderr = proc.communicate()
289 self.assertEqual(proc.returncode, 0)
290 os__file__, os__cached__ = stdout.split()
291 self.assertTrue(os.path.isabs(os__file__))
292 self.assertTrue(os.path.isabs(os__cached__))
274293
275 def test_no_duplicate_paths(self):294 def test_no_duplicate_paths(self):
276 # No duplicate paths should exist in sys.path295 # No duplicate paths should exist in sys.path
277296
=== modified file 'Lib/test/test_zipfile.py'
--- Lib/test/test_zipfile.py 2010-02-07 20:24:02 +0000
+++ Lib/test/test_zipfile.py 2010-04-16 18:55:41 +0000
@@ -6,6 +6,7 @@
66
7import io7import io
8import os8import os
9import imp
9import time10import time
10import shutil11import shutil
11import struct12import struct
@@ -587,7 +588,13 @@
587 with zipfile.PyZipFile(TemporaryFile(), "w") as zipfp:588 with zipfile.PyZipFile(TemporaryFile(), "w") as zipfp:
588 fn = __file__589 fn = __file__
589 if fn.endswith('.pyc') or fn.endswith('.pyo'):590 if fn.endswith('.pyc') or fn.endswith('.pyo'):
590 fn = fn[:-1]591 path_split = fn.split(os.sep)
592 if os.altsep is not None:
593 path_split.extend(fn.split(os.altsep))
594 if '__pycache__' in path_split:
595 fn = imp.source_from_cache(fn)
596 else:
597 fn = fn[:-1]
591598
592 zipfp.writepy(fn)599 zipfp.writepy(fn)
593600
594601
=== modified file 'Lib/test/test_zipimport.py'
--- Lib/test/test_zipimport.py 2010-03-14 10:23:39 +0000
+++ Lib/test/test_zipimport.py 2010-04-16 18:55:41 +0000
@@ -48,17 +48,14 @@
48test_pyc = make_pyc(test_co, NOW)48test_pyc = make_pyc(test_co, NOW)
4949
5050
51if __debug__:
52 pyc_ext = ".pyc"
53else:
54 pyc_ext = ".pyo"
55
56
57TESTMOD = "ziptestmodule"51TESTMOD = "ziptestmodule"
58TESTPACK = "ziptestpackage"52TESTPACK = "ziptestpackage"
59TESTPACK2 = "ziptestpackage2"53TESTPACK2 = "ziptestpackage2"
60TEMP_ZIP = os.path.abspath("junk95142.zip")54TEMP_ZIP = os.path.abspath("junk95142.zip")
6155
56pyc_file = imp.cache_from_source(TESTMOD + '.py')
57pyc_ext = ('.pyc' if __debug__ else '.pyo')
58
6259
63class UncompressedZipImportTestCase(ImportHooksBaseTestCase):60class UncompressedZipImportTestCase(ImportHooksBaseTestCase):
6461
@@ -83,14 +80,11 @@
83 stuff = kw.get("stuff", None)80 stuff = kw.get("stuff", None)
84 if stuff is not None:81 if stuff is not None:
85 # Prepend 'stuff' to the start of the zipfile82 # Prepend 'stuff' to the start of the zipfile
86 f = open(TEMP_ZIP, "rb")83 with open(TEMP_ZIP, "rb") as f:
87 data = f.read()84 data = f.read()
88 f.close()85 with open(TEMP_ZIP, "wb") as f:
8986 f.write(stuff)
90 f = open(TEMP_ZIP, "wb")87 f.write(data)
91 f.write(stuff)
92 f.write(data)
93 f.close()
9488
95 sys.path.insert(0, TEMP_ZIP)89 sys.path.insert(0, TEMP_ZIP)
9690
@@ -180,8 +174,9 @@
180174
181 def testBadMTime(self):175 def testBadMTime(self):
182 badtime_pyc = bytearray(test_pyc)176 badtime_pyc = bytearray(test_pyc)
183 badtime_pyc[7] ^= 0x02 # flip the second bit -- not the first as that one177 # flip the second bit -- not the first as that one isn't stored in the
184 # isn't stored in the .py's mtime in the zip archive.178 # .py's mtime in the zip archive.
179 badtime_pyc[7] ^= 0x02
185 files = {TESTMOD + ".py": (NOW, test_src),180 files = {TESTMOD + ".py": (NOW, test_src),
186 TESTMOD + pyc_ext: (NOW, badtime_pyc)}181 TESTMOD + pyc_ext: (NOW, badtime_pyc)}
187 self.doTest(".py", files, TESTMOD)182 self.doTest(".py", files, TESTMOD)
@@ -232,7 +227,8 @@
232 self.assertEquals(zi.get_source(TESTPACK), None)227 self.assertEquals(zi.get_source(TESTPACK), None)
233 self.assertEquals(zi.get_source(mod_path), None)228 self.assertEquals(zi.get_source(mod_path), None)
234 self.assertEquals(zi.get_filename(mod_path), mod.__file__)229 self.assertEquals(zi.get_filename(mod_path), mod.__file__)
235 # To pass in the module name instead of the path, we must use the right importer230 # To pass in the module name instead of the path, we must use the
231 # right importer
236 loader = mod.__loader__232 loader = mod.__loader__
237 self.assertEquals(loader.get_source(mod_name), None)233 self.assertEquals(loader.get_source(mod_name), None)
238 self.assertEquals(loader.get_filename(mod_name), mod.__file__)234 self.assertEquals(loader.get_filename(mod_name), mod.__file__)
@@ -266,8 +262,10 @@
266 mod = zi.load_module(TESTPACK2)262 mod = zi.load_module(TESTPACK2)
267 self.assertEquals(zi.get_filename(TESTPACK2), mod.__file__)263 self.assertEquals(zi.get_filename(TESTPACK2), mod.__file__)
268264
269 self.assertEquals(zi.is_package(TESTPACK2 + os.sep + '__init__'), False)265 self.assertEquals(
270 self.assertEquals(zi.is_package(TESTPACK2 + os.sep + TESTMOD), False)266 zi.is_package(TESTPACK2 + os.sep + '__init__'), False)
267 self.assertEquals(
268 zi.is_package(TESTPACK2 + os.sep + TESTMOD), False)
271269
272 mod_path = TESTPACK2 + os.sep + TESTMOD270 mod_path = TESTPACK2 + os.sep + TESTMOD
273 mod_name = module_path_to_dotted_name(mod_path)271 mod_name = module_path_to_dotted_name(mod_path)
@@ -276,7 +274,8 @@
276 self.assertEquals(zi.get_source(TESTPACK2), None)274 self.assertEquals(zi.get_source(TESTPACK2), None)
277 self.assertEquals(zi.get_source(mod_path), None)275 self.assertEquals(zi.get_source(mod_path), None)
278 self.assertEquals(zi.get_filename(mod_path), mod.__file__)276 self.assertEquals(zi.get_filename(mod_path), mod.__file__)
279 # To pass in the module name instead of the path, we must use the right importer277 # To pass in the module name instead of the path, we must use the
278 # right importer
280 loader = mod.__loader__279 loader = mod.__loader__
281 self.assertEquals(loader.get_source(mod_name), None)280 self.assertEquals(loader.get_source(mod_name), None)
282 self.assertEquals(loader.get_filename(mod_name), mod.__file__)281 self.assertEquals(loader.get_filename(mod_name), mod.__file__)
283282
=== modified file 'Lib/zipfile.py'
--- Lib/zipfile.py 2010-02-07 20:24:02 +0000
+++ Lib/zipfile.py 2010-04-16 18:55:41 +0000
@@ -3,10 +3,17 @@
33
4XXX references to utf-8 need further investigation.4XXX references to utf-8 need further investigation.
5"""5"""
6import struct, os, time, sys, shutil
7import binascii, io, stat
8import io6import io
7import os
9import re8import re
9import imp
10import sys
11import time
12import stat
13import shutil
14import struct
15import binascii
16
1017
11try:18try:
12 import zlib # We may need its compression method19 import zlib # We may need its compression method
@@ -1303,22 +1310,42 @@
1303 file_py = pathname + ".py"1310 file_py = pathname + ".py"
1304 file_pyc = pathname + ".pyc"1311 file_pyc = pathname + ".pyc"
1305 file_pyo = pathname + ".pyo"1312 file_pyo = pathname + ".pyo"
1306 if os.path.isfile(file_pyo) and \1313 pycache_pyc = imp.cache_from_source(file_py, True)
1307 os.stat(file_pyo).st_mtime >= os.stat(file_py).st_mtime:1314 pycache_pyo = imp.cache_from_source(file_py, False)
1308 fname = file_pyo # Use .pyo file1315 if (os.path.isfile(file_pyo) and
1309 elif not os.path.isfile(file_pyc) or \1316 os.stat(file_pyo).st_mtime >= os.stat(file_py).st_mtime):
1310 os.stat(file_pyc).st_mtime < os.stat(file_py).st_mtime:1317 # Use .pyo file.
1318 arcname = fname = file_pyo
1319 elif (os.path.isfile(file_pyc) and
1320 os.stat(file_pyc).st_mtime >= os.stat(file_py).st_mtime):
1321 # Use .pyc file.
1322 arcname = fname = file_pyc
1323 elif (os.path.isfile(pycache_pyc) and
1324 os.stat(pycache_pyc).st_mtime >= os.stat(file_py).st_mtime):
1325 # Use the __pycache__/*.pyc file, but write it to the legacy pyc
1326 # file name in the archive.
1327 fname = pycache_pyc
1328 arcname = file_pyc
1329 elif (os.path.isfile(pycache_pyo) and
1330 os.stat(pycache_pyo).st_mtime >= os.stat(file_py).st_mtime):
1331 # Use the __pycache__/*.pyo file, but write it to the legacy pyo
1332 # file name in the archive.
1333 fname = pycache_pyo
1334 arcname = file_pyo
1335 else:
1336 # Compile py into PEP 3147 pyc file.
1311 import py_compile1337 import py_compile
1312 if self.debug:1338 if self.debug:
1313 print("Compiling", file_py)1339 print("Compiling", file_py)
1314 try:1340 try:
1315 py_compile.compile(file_py, file_pyc, None, True)1341 py_compile.compile(file_py, doraise=True)
1316 except py_compile.PyCompileError as err:1342 except py_compile.PyCompileError as error:
1317 print(err.msg)1343 print(err.msg)
1318 fname = file_pyc1344 fname = file_py
1319 else:1345 else:
1320 fname = file_pyc1346 fname = (pycache_pyc if __debug__ else pycache_pyo)
1321 archivename = os.path.split(fname)[1]1347 arcname = (file_pyc if __debug__ else file_pyo)
1348 archivename = os.path.split(arcname)[1]
1322 if basename:1349 if basename:
1323 archivename = "%s/%s" % (basename, archivename)1350 archivename = "%s/%s" % (basename, archivename)
1324 return (fname, archivename)1351 return (fname, archivename)
13251352
=== modified file 'Makefile.pre.in'
--- Makefile.pre.in 2010-04-11 23:16:50 +0000
+++ Makefile.pre.in 2010-04-16 18:55:41 +0000
@@ -1161,6 +1161,7 @@
1161# files, which clobber removes as well1161# files, which clobber removes as well
1162pycremoval:1162pycremoval:
1163 find $(srcdir) -name '*.py[co]' -exec rm -f {} ';'1163 find $(srcdir) -name '*.py[co]' -exec rm -f {} ';'
1164 find $(srcdir) -name '__pycache__' | xargs rmdir
11641165
1165rmtestturds:1166rmtestturds:
1166 -rm -f *BAD *GOOD *SKIPPED1167 -rm -f *BAD *GOOD *SKIPPED
11671168
=== modified file 'Python/import.c'
--- Python/import.c 2010-03-25 23:30:20 +0000
+++ Python/import.c 2010-04-16 18:55:41 +0000
@@ -43,6 +43,15 @@
43 The current working scheme is to increment the previous value by43 The current working scheme is to increment the previous value by
44 10.44 10.
4545
46 Starting with the adoption of PEP 3147 in Python 3.2, every bump in magic
47 number also includes a new "magic tag", i.e. a human readable string used
48 to represent the magic number in __pycache__ directories. When you change
49 the magic number, you must also set a new unique magic tag. Generally this
50 can be named after the Python major version of the magic number bump, but
51 it can really be anything, as long as it's different than anything else
52 that's come before. The tags are included in the following table, starting
53 with Python 3.2a0.
54
46 Known values:55 Known values:
47 Python 1.5: 2012156 Python 1.5: 20121
48 Python 1.5.1: 2012157 Python 1.5.1: 20121
@@ -91,11 +100,18 @@
91 Python 3.1a0: 3151 (optimize conditional branches:100 Python 3.1a0: 3151 (optimize conditional branches:
92 introduce POP_JUMP_IF_FALSE and POP_JUMP_IF_TRUE)101 introduce POP_JUMP_IF_FALSE and POP_JUMP_IF_TRUE)
93 Python 3.2a0: 3160 (add SETUP_WITH)102 Python 3.2a0: 3160 (add SETUP_WITH)
103 tag: cpython-32
94*/104*/
95105
106/* If you change MAGIC, you must change TAG and you must insert the old value
107 into _PyMagicNumberTags below.
108*/
96#define MAGIC (3160 | ((long)'\r'<<16) | ((long)'\n'<<24))109#define MAGIC (3160 | ((long)'\r'<<16) | ((long)'\n'<<24))
97/* Magic word as global */110#define TAG "cpython-32"
111#define CACHEDIR "__pycache__"
112/* Current magic word and string tag as globals. */
98static long pyc_magic = MAGIC;113static long pyc_magic = MAGIC;
114static const char *pyc_tag = TAG;
99115
100/* See _PyImport_FixupExtension() below */116/* See _PyImport_FixupExtension() below */
101static PyObject *extensions = NULL;117static PyObject *extensions = NULL;
@@ -517,7 +533,7 @@
517}533}
518534
519535
520/* Helper for pythonrun.c -- return magic number */536/* Helper for pythonrun.c -- return magic number and tag. */
521537
522long538long
523PyImport_GetMagicNumber(void)539PyImport_GetMagicNumber(void)
@@ -526,6 +542,12 @@
526}542}
527543
528544
545const char *
546PyImport_GetMagicTag(void)
547{
548 return pyc_tag;
549}
550
529/* Magic for extension modules (built-in as well as dynamically551/* Magic for extension modules (built-in as well as dynamically
530 loaded). To prevent initializing an extension module more than552 loaded). To prevent initializing an extension module more than
531 once, we keep a static dictionary 'extensions' keyed by module name553 once, we keep a static dictionary 'extensions' keyed by module name
@@ -671,7 +693,10 @@
671 "sys.modules failed");693 "sys.modules failed");
672}694}
673695
674static PyObject * get_sourcefile(const char *file);696static PyObject * get_sourcefile(char *file);
697static char *make_source_pathname(char *pathname, char *buf);
698static char *make_compiled_pathname(char *pathname, char *buf, size_t buflen,
699 int debug);
675700
676/* Execute a code object in a module and return the module object701/* Execute a code object in a module and return the module object
677 * WITH INCREMENTED REFERENCE COUNT. If an error occurs, name is702 * WITH INCREMENTED REFERENCE COUNT. If an error occurs, name is
@@ -679,16 +704,28 @@
679 * in sys.modules. The caller may wish to restore the original704 * in sys.modules. The caller may wish to restore the original
680 * module object (if any) in this case; PyImport_ReloadModule is an705 * module object (if any) in this case; PyImport_ReloadModule is an
681 * example.706 * example.
707 *
708 * Note that PyImport_ExecCodeModuleWithPathnames() is the preferred, richer
709 * interface. The other two exist primarily for backward compatibility.
682 */710 */
683PyObject *711PyObject *
684PyImport_ExecCodeModule(char *name, PyObject *co)712PyImport_ExecCodeModule(char *name, PyObject *co)
685{713{
686 return PyImport_ExecCodeModuleEx(name, co, (char *)NULL);714 return PyImport_ExecCodeModuleWithPathnames(
715 name, co, (char *)NULL, (char *)NULL);
687}716}
688717
689PyObject *718PyObject *
690PyImport_ExecCodeModuleEx(char *name, PyObject *co, char *pathname)719PyImport_ExecCodeModuleEx(char *name, PyObject *co, char *pathname)
691{720{
721 return PyImport_ExecCodeModuleWithPathnames(
722 name, co, pathname, (char *)NULL);
723}
724
725PyObject *
726PyImport_ExecCodeModuleWithPathnames(char *name, PyObject *co, char *pathname,
727 char *cpathname)
728{
692 PyObject *modules = PyImport_GetModuleDict();729 PyObject *modules = PyImport_GetModuleDict();
693 PyObject *m, *d, *v;730 PyObject *m, *d, *v;
694731
@@ -718,6 +755,20 @@
718 PyErr_Clear(); /* Not important enough to report */755 PyErr_Clear(); /* Not important enough to report */
719 Py_DECREF(v);756 Py_DECREF(v);
720757
758 /* Remember the pyc path name as the __cached__ attribute. */
759 if (cpathname == NULL) {
760 v = Py_None;
761 Py_INCREF(v);
762 }
763 else if ((v = PyUnicode_FromString(cpathname)) == NULL) {
764 PyErr_Clear(); /* Not important enough to report */
765 v = Py_None;
766 Py_INCREF(v);
767 }
768 if (PyDict_SetItemString(d, "__cached__", v) != 0)
769 PyErr_Clear(); /* Not important enough to report */
770 Py_DECREF(v);
771
721 v = PyEval_EvalCode((PyCodeObject *)co, d, d);772 v = PyEval_EvalCode((PyCodeObject *)co, d, d);
722 if (v == NULL)773 if (v == NULL)
723 goto error;774 goto error;
@@ -740,31 +791,188 @@
740}791}
741792
742793
794/* Like strrchr(string, '/') but searches for the rightmost of either SEP
795 or ALTSEP, if the latter is defined.
796*/
797static char *
798rightmost_sep(char *s)
799{
800 char *found, c;
801 for (found = NULL; (c = *s); s++) {
802 if (c == SEP
803#ifdef ALTSEP
804 || c == ALTSEP
805#endif
806 )
807 {
808 found = s;
809 }
810 }
811 return found;
812}
813
814
743/* Given a pathname for a Python source file, fill a buffer with the815/* Given a pathname for a Python source file, fill a buffer with the
744 pathname for the corresponding compiled file. Return the pathname816 pathname for the corresponding compiled file. Return the pathname
745 for the compiled file, or NULL if there's no space in the buffer.817 for the compiled file, or NULL if there's no space in the buffer.
746 Doesn't set an exception. */818 Doesn't set an exception. */
747819
748static char *820static char *
749make_compiled_pathname(char *pathname, char *buf, size_t buflen)821make_compiled_pathname(char *pathname, char *buf, size_t buflen, int debug)
750{822{
823 /* foo.py -> __pycache__/foo.<tag>.pyc */
751 size_t len = strlen(pathname);824 size_t len = strlen(pathname);
752 if (len+2 > buflen)825 size_t i, save;
826 char *pos;
827 int sep = SEP;
828
829 /* Sanity check that the buffer has roughly enough space to hold what
830 will eventually be the full path to the compiled file. The 5 extra
831 bytes include the slash afer __pycache__, the two extra dots, the
832 extra trailing character ('c' or 'o') and null. This isn't exact
833 because the contents of the buffer can affect how many actual
834 characters of the string get into the buffer. We'll do a final
835 sanity check before writing the extension to ensure we do not
836 overflow the buffer.
837 */
838 if (len + strlen(CACHEDIR) + strlen(pyc_tag) + 5 > buflen)
753 return NULL;839 return NULL;
754840
755#ifdef MS_WINDOWS841 /* Find the last path separator and copy everything from the start of
756 /* Treat .pyw as if it were .py. The case of ".pyw" must match842 the source string up to and including the separator.
757 that used in _PyImport_StandardFiletab. */843 */
758 if (len >= 4 && strcmp(&pathname[len-4], ".pyw") == 0)844 if ((pos = rightmost_sep(pathname)) == NULL) {
759 --len; /* pretend 'w' isn't there */845 i = 0;
846 }
847 else {
848 sep = *pos;
849 i = pos - pathname + 1;
850 strncpy(buf, pathname, i);
851 }
852
853 save = i;
854 buf[i++] = '\0';
855 /* Add __pycache__/ */
856 strcat(buf, CACHEDIR);
857 i += strlen(CACHEDIR) - 1;
858 buf[i++] = sep;
859 buf[i++] = '\0';
860 /* Add the base filename, but remove the .py or .pyw extension, since
861 the tag name must go before the extension.
862 */
863 strcat(buf, pathname + save);
864 if ((pos = strrchr(buf, '.')) != NULL)
865 *++pos = '\0';
866 strcat(buf, pyc_tag);
867 /* The length test above assumes that we're only adding one character
868 to the end of what would normally be the extension. What if there
869 is no extension, or the string ends in '.' or '.p', and otherwise
870 fills the buffer? By appending 4 more characters onto the string
871 here, we could overrun the buffer.
872
873 As a simple example, let's say buflen=32 and the input string is
874 'xxx.py'. strlen() would be 6 and the test above would yield:
875
876 (6 + 11 + 10 + 5 == 32) > 32
877
878 which is false and so the name mangling would continue. This would
879 be fine because we'd end up with this string in buf:
880
881 __pycache__/xxx.cpython-32.pyc\0
882
883 strlen(of that) == 30 + the nul fits inside a 32 character buffer.
884 We can even handle an input string of say 'xxxxx' above because
885 that's (5 + 11 + 10 + 5 == 31) > 32 which is also false. Name
886 mangling that yields:
887
888 __pycache__/xxxxxcpython-32.pyc\0
889
890 which is 32 characters including the nul, and thus fits in the
891 buffer. However, an input string of 'xxxxxx' would yield a result
892 string of:
893
894 __pycache__/xxxxxxcpython-32.pyc\0
895
896 which is 33 characters long (including the nul), thus overflowing
897 the buffer, even though the first test would fail, i.e.: the input
898 string is also 6 characters long, so 32 > 32 is false.
899
900 The reason the first test fails but we still overflow the buffer is
901 that the test above only expects to add one extra character to be
902 added to the extension, and here we're adding three (pyc). We
903 don't add the first dot, so that reclaims one of expected
904 positions, leaving us overflowing by 1 byte (3 extra - 1 reclaimed
905 dot - 1 expected extra == 1 overflowed).
906
907 The best we can do is ensure that we still have enough room in the
908 target buffer before we write the extension. Because it's always
909 only the extension that can cause the overflow, and never the other
910 path bytes we've written, it's sufficient to just do one more test
911 here. Still, the assertion that follows can't hurt.
912 */
913#if 0
914 printf("strlen(buf): %d; buflen: %d\n", (int)strlen(buf), (int)buflen);
760#endif915#endif
761 memcpy(buf, pathname, len);916 if (strlen(buf) + 5 > buflen)
762 buf[len] = Py_OptimizeFlag ? 'o' : 'c';917 return NULL;
763 buf[len+1] = '\0';918 strcat(buf, debug ? ".pyc" : ".pyo");
764919 assert(strlen(buf) < buflen);
765 return buf;920 return buf;
766}921}
767922
923
924/* Given a pathname to a Python byte compiled file, return the path to the
925 source file, if the path matches the PEP 3147 format. This does not check
926 for any file existence, however, if the pyc file name does not match PEP
927 3147 style, NULL is returned. buf must be at least as big as pathname;
928 the resulting path will always be shorter. */
929
930static char *
931make_source_pathname(char *pathname, char *buf)
932{
933 /* __pycache__/foo.<tag>.pyc -> foo.py */
934 size_t i, j;
935 char *left, *right, *dot0, *dot1, sep;
936
937 /* Look back two slashes from the end. In between these two slashes
938 must be the string __pycache__ or this is not a PEP 3147 style
939 path. It's possible for there to be only one slash.
940 */
941 if ((right = rightmost_sep(pathname)) == NULL)
942 return NULL;
943 sep = *right;
944 *right = '\0';
945 left = rightmost_sep(pathname);
946 *right = sep;
947 if (left == NULL)
948 left = pathname;
949 else
950 left++;
951 if (right-left != strlen(CACHEDIR) ||
952 strncmp(left, CACHEDIR, right-left) != 0)
953 return NULL;
954
955 /* Now verify that the path component to the right of the last slash
956 has two dots in it.
957 */
958 if ((dot0 = strchr(right + 1, '.')) == NULL)
959 return NULL;
960 if ((dot1 = strchr(dot0 + 1, '.')) == NULL)
961 return NULL;
962 /* Too many dots? */
963 if (strchr(dot1 + 1, '.') != NULL)
964 return NULL;
965
966 /* This is a PEP 3147 path. Start by copying everything from the
967 start of pathname up to and including the leftmost slash. Then
968 copy the file's basename, removing the magic tag and adding a .py
969 suffix.
970 */
971 strncpy(buf, pathname, (i=left-pathname));
972 strncpy(buf+i, right+1, (j=dot0-right));
973 strcpy(buf+i+j, "py");
974 return buf;
975}
768976
769/* Given a pathname for a Python source file, its time of last977/* Given a pathname for a Python source file, its time of last
770 modification, and a pathname for a compiled file, check whether the978 modification, and a pathname for a compiled file, check whether the
@@ -846,7 +1054,8 @@
846 if (Py_VerboseFlag)1054 if (Py_VerboseFlag)
847 PySys_WriteStderr("import %s # precompiled from %s\n",1055 PySys_WriteStderr("import %s # precompiled from %s\n",
848 name, cpathname);1056 name, cpathname);
849 m = PyImport_ExecCodeModuleEx(name, (PyObject *)co, cpathname);1057 m = PyImport_ExecCodeModuleWithPathnames(
1058 name, (PyObject *)co, cpathname, cpathname);
850 Py_DECREF(co);1059 Py_DECREF(co);
8511060
852 return m;1061 return m;
@@ -919,12 +1128,41 @@
919write_compiled_module(PyCodeObject *co, char *cpathname, struct stat *srcstat)1128write_compiled_module(PyCodeObject *co, char *cpathname, struct stat *srcstat)
920{1129{
921 FILE *fp;1130 FILE *fp;
1131 char *dirpath;
922 time_t mtime = srcstat->st_mtime;1132 time_t mtime = srcstat->st_mtime;
923#ifdef MS_WINDOWS /* since Windows uses different permissions */1133#ifdef MS_WINDOWS /* since Windows uses different permissions */
924 mode_t mode = srcstat->st_mode & ~S_IEXEC;1134 mode_t mode = srcstat->st_mode & ~S_IEXEC;
1135 mode_t dirmode = srcstat->st_mode | S_IEXEC; /* XXX Is this correct
1136 for Windows?
1137 2010-04-07 BAW */
925#else1138#else
926 mode_t mode = srcstat->st_mode & ~S_IXUSR & ~S_IXGRP & ~S_IXOTH;1139 mode_t mode = srcstat->st_mode & ~S_IXUSR & ~S_IXGRP & ~S_IXOTH;
1140 mode_t dirmode = (srcstat->st_mode |
1141 S_IXUSR | S_IXGRP | S_IXOTH |
1142 S_IWUSR | S_IWGRP | S_IWOTH);
927#endif1143#endif
1144 int saved;
1145
1146 /* Ensure that the __pycache__ directory exists. */
1147 dirpath = rightmost_sep(cpathname);
1148 if (dirpath == NULL) {
1149 if (Py_VerboseFlag)
1150 PySys_WriteStderr(
1151 "# no %s path found %s\n",
1152 CACHEDIR, cpathname);
1153 return;
1154 }
1155 saved = *dirpath;
1156 *dirpath = '\0';
1157 /* XXX call os.mkdir() or maybe CreateDirectoryA() on Windows? */
1158 if (mkdir(cpathname, dirmode) < 0 && errno != EEXIST) {
1159 *dirpath = saved;
1160 if (Py_VerboseFlag)
1161 PySys_WriteStderr(
1162 "# cannot create cache dir %s\n", cpathname);
1163 return;
1164 }
1165 *dirpath = saved;
9281166
929 fp = open_exclusive(cpathname, mode);1167 fp = open_exclusive(cpathname, mode);
930 if (fp == NULL) {1168 if (fp == NULL) {
@@ -1032,8 +1270,8 @@
1032 return NULL;1270 return NULL;
1033 }1271 }
1034#endif1272#endif
1035 cpathname = make_compiled_pathname(pathname, buf,1273 cpathname = make_compiled_pathname(
1036 (size_t)MAXPATHLEN + 1);1274 pathname, buf, (size_t)MAXPATHLEN + 1, !Py_OptimizeFlag);
1037 if (cpathname != NULL &&1275 if (cpathname != NULL &&
1038 (fpc = check_compiled_module(pathname, st.st_mtime, cpathname))) {1276 (fpc = check_compiled_module(pathname, st.st_mtime, cpathname))) {
1039 co = read_compiled_module(cpathname, fpc);1277 co = read_compiled_module(cpathname, fpc);
@@ -1060,7 +1298,8 @@
1060 write_compiled_module(co, cpathname, &st);1298 write_compiled_module(co, cpathname, &st);
1061 }1299 }
1062 }1300 }
1063 m = PyImport_ExecCodeModuleEx(name, (PyObject *)co, pathname);1301 m = PyImport_ExecCodeModuleWithPathnames(
1302 name, (PyObject *)co, pathname, cpathname);
1064 Py_DECREF(co);1303 Py_DECREF(co);
10651304
1066 return m;1305 return m;
@@ -1070,7 +1309,7 @@
1070 * Returns the path to the py file if available, else the given path1309 * Returns the path to the py file if available, else the given path
1071 */1310 */
1072static PyObject *1311static PyObject *
1073get_sourcefile(const char *file)1312get_sourcefile(char *file)
1074{1313{
1075 char py[MAXPATHLEN + 1];1314 char py[MAXPATHLEN + 1];
1076 Py_ssize_t len;1315 Py_ssize_t len;
@@ -1087,8 +1326,15 @@
1087 return PyUnicode_DecodeFSDefault(file);1326 return PyUnicode_DecodeFSDefault(file);
1088 }1327 }
10891328
1090 strncpy(py, file, len-1);1329 /* Start by trying to turn PEP 3147 path into source path. If that
1091 py[len-1] = '\0';1330 * fails, just chop off the trailing character, i.e. legacy pyc path
1331 * to py.
1332 */
1333 if (make_source_pathname(file, py) == NULL) {
1334 strncpy(py, file, len-1);
1335 py[len-1] = '\0';
1336 }
1337
1092 if (stat(py, &statbuf) == 0 &&1338 if (stat(py, &statbuf) == 0 &&
1093 S_ISREG(statbuf.st_mode)) {1339 S_ISREG(statbuf.st_mode)) {
1094 u = PyUnicode_DecodeFSDefault(py);1340 u = PyUnicode_DecodeFSDefault(py);
@@ -2813,16 +3059,28 @@
2813*/3059*/
28143060
2815static PyObject *3061static PyObject *
2816imp_get_magic(PyObject *self, PyObject *noargs)3062imp_make_magic(long magic)
2817{3063{
2818 char buf[4];3064 char buf[4];
28193065
2820 buf[0] = (char) ((pyc_magic >> 0) & 0xff);3066 buf[0] = (char) ((magic >> 0) & 0xff);
2821 buf[1] = (char) ((pyc_magic >> 8) & 0xff);3067 buf[1] = (char) ((magic >> 8) & 0xff);
2822 buf[2] = (char) ((pyc_magic >> 16) & 0xff);3068 buf[2] = (char) ((magic >> 16) & 0xff);
2823 buf[3] = (char) ((pyc_magic >> 24) & 0xff);3069 buf[3] = (char) ((magic >> 24) & 0xff);
28243070
2825 return PyBytes_FromStringAndSize(buf, 4);3071 return PyBytes_FromStringAndSize(buf, 4);
3072};
3073
3074static PyObject *
3075imp_get_magic(PyObject *self, PyObject *noargs)
3076{
3077 return imp_make_magic(pyc_magic);
3078}
3079
3080static PyObject *
3081imp_get_tag(PyObject *self, PyObject *noargs)
3082{
3083 return PyUnicode_FromString(pyc_tag);
2826}3084}
28273085
2828static PyObject *3086static PyObject *
@@ -3190,6 +3448,75 @@
3190\n\3448\n\
3191Reload the module. The module must have been successfully imported before.");3449Reload the module. The module must have been successfully imported before.");
31923450
3451static PyObject *
3452imp_cache_from_source(PyObject *self, PyObject *args, PyObject *kws)
3453{
3454 static char *kwlist[] = {"path", "debug_override", NULL};
3455
3456 char buf[MAXPATHLEN+1];
3457 char *pathname, *cpathname;
3458 PyObject *debug_override = Py_None;
3459 int debug = !Py_OptimizeFlag;
3460
3461 if (!PyArg_ParseTupleAndKeywords(
3462 args, kws, "es|O", kwlist,
3463 Py_FileSystemDefaultEncoding, &pathname, &debug_override))
3464 return NULL;
3465
3466 if (debug_override != Py_None)
3467 if ((debug = PyObject_IsTrue(debug_override)) < 0)
3468 return NULL;
3469
3470 cpathname = make_compiled_pathname(pathname, buf, MAXPATHLEN+1, debug);
3471 PyMem_Free(pathname);
3472
3473 if (cpathname == NULL) {
3474 PyErr_Format(PyExc_SystemError, "path buffer too short");
3475 return NULL;
3476 }
3477 return PyUnicode_FromString(buf);
3478}
3479
3480PyDoc_STRVAR(doc_cache_from_source,
3481"Given the path to a .py file, return the path to its .pyc/.pyo file.\n\
3482\n\
3483The .py file does not need to exist; this simply returns the path to the\n\
3484.pyc/.pyo file calculated as if the .py file were imported. The extension\n\
3485will be .pyc unless __debug__ is not defined, then it will be .pyo.\n\
3486\n\
3487If debug_override is not None, then it must be a boolean and is taken as\n\
3488the value of __debug__ instead.");
3489
3490static PyObject *
3491imp_source_from_cache(PyObject *self, PyObject *args, PyObject *kws)
3492{
3493 static char *kwlist[] = {"path", NULL};
3494
3495 char *pathname;
3496 char buf[MAXPATHLEN+1];
3497
3498 if (!PyArg_ParseTupleAndKeywords(
3499 args, kws, "es", kwlist,
3500 Py_FileSystemDefaultEncoding, &pathname))
3501 return NULL;
3502
3503 if (make_source_pathname(pathname, buf) == NULL) {
3504 PyErr_Format(PyExc_ValueError, "Not a PEP 3147 pyc path: %s",
3505 pathname);
3506 PyMem_Free(pathname);
3507 return NULL;
3508 }
3509 PyMem_Free(pathname);
3510 return PyUnicode_FromString(buf);
3511}
3512
3513PyDoc_STRVAR(doc_source_from_cache,
3514"Given the path to a .pyc./.pyo file, return the path to its .py file.\n\
3515\n\
3516The .pyc/.pyo file does not need to exist; this simply returns the path to\n\
3517the .py file calculated to correspond to the .pyc/.pyo file. If path\n\
3518does not conform to PEP 3147 format, ValueError will be raised.");
3519
3193/* Doc strings */3520/* Doc strings */
31943521
3195PyDoc_STRVAR(doc_imp,3522PyDoc_STRVAR(doc_imp,
@@ -3212,6 +3539,10 @@
3212"get_magic() -> string\n\3539"get_magic() -> string\n\
3213Return the magic number for .pyc or .pyo files.");3540Return the magic number for .pyc or .pyo files.");
32143541
3542PyDoc_STRVAR(doc_get_tag,
3543"get_tag() -> string\n\
3544Return the magic tag for .pyc or .pyo files.");
3545
3215PyDoc_STRVAR(doc_get_suffixes,3546PyDoc_STRVAR(doc_get_suffixes,
3216"get_suffixes() -> [(suffix, mode, type), ...]\n\3547"get_suffixes() -> [(suffix, mode, type), ...]\n\
3217Return a list of (suffix, mode, type) tuples describing the files\n\3548Return a list of (suffix, mode, type) tuples describing the files\n\
@@ -3242,6 +3573,7 @@
3242static PyMethodDef imp_methods[] = {3573static PyMethodDef imp_methods[] = {
3243 {"find_module", imp_find_module, METH_VARARGS, doc_find_module},3574 {"find_module", imp_find_module, METH_VARARGS, doc_find_module},
3244 {"get_magic", imp_get_magic, METH_NOARGS, doc_get_magic},3575 {"get_magic", imp_get_magic, METH_NOARGS, doc_get_magic},
3576 {"get_tag", imp_get_tag, METH_NOARGS, doc_get_tag},
3245 {"get_suffixes", imp_get_suffixes, METH_NOARGS, doc_get_suffixes},3577 {"get_suffixes", imp_get_suffixes, METH_NOARGS, doc_get_suffixes},
3246 {"load_module", imp_load_module, METH_VARARGS, doc_load_module},3578 {"load_module", imp_load_module, METH_VARARGS, doc_load_module},
3247 {"new_module", imp_new_module, METH_VARARGS, doc_new_module},3579 {"new_module", imp_new_module, METH_VARARGS, doc_new_module},
@@ -3249,6 +3581,10 @@
3249 {"acquire_lock", imp_acquire_lock, METH_NOARGS, doc_acquire_lock},3581 {"acquire_lock", imp_acquire_lock, METH_NOARGS, doc_acquire_lock},
3250 {"release_lock", imp_release_lock, METH_NOARGS, doc_release_lock},3582 {"release_lock", imp_release_lock, METH_NOARGS, doc_release_lock},
3251 {"reload", imp_reload, METH_O, doc_reload},3583 {"reload", imp_reload, METH_O, doc_reload},
3584 {"cache_from_source", (PyCFunction)imp_cache_from_source,
3585 METH_VARARGS | METH_KEYWORDS, doc_cache_from_source},
3586 {"source_from_cache", (PyCFunction)imp_source_from_cache,
3587 METH_VARARGS | METH_KEYWORDS, doc_source_from_cache},
3252 /* The rest are obsolete */3588 /* The rest are obsolete */
3253 {"get_frozen_object", imp_get_frozen_object, METH_VARARGS},3589 {"get_frozen_object", imp_get_frozen_object, METH_VARARGS},
3254 {"is_frozen_package", imp_is_frozen_package, METH_VARARGS},3590 {"is_frozen_package", imp_is_frozen_package, METH_VARARGS},
@@ -3436,7 +3772,6 @@
3436 failure:3772 failure:
3437 Py_XDECREF(m);3773 Py_XDECREF(m);
3438 return NULL;3774 return NULL;
3439
3440}3775}
34413776
34423777
34433778
=== modified file 'Python/pythonrun.c'
--- Python/pythonrun.c 2010-03-14 06:49:55 +0000
+++ Python/pythonrun.c 2010-04-16 18:55:41 +0000
@@ -1155,6 +1155,8 @@
1155 Py_DECREF(f);1155 Py_DECREF(f);
1156 return -1;1156 return -1;
1157 }1157 }
1158 if (PyDict_SetItemString(d, "__cached__", Py_None) < 0)
1159 return -1;
1158 set_file_name = 1;1160 set_file_name = 1;
1159 Py_DECREF(f);1161 Py_DECREF(f);
1160 }1162 }

Subscribers

People subscribed via source and target branches

to all changes: