Merge lp:~ipython-dev/ipython/kernel-config into lp:ipython/0.11
- kernel-config
- Merge into trunk
Status: | Merged |
---|---|
Merged at revision: | not available |
Proposed branch: | lp:~ipython-dev/ipython/kernel-config |
Merge into: | lp:ipython/0.11 |
Diff against target: |
11644 lines (+6503/-3129) 79 files modified
IPython/config/api.py (+0/-102) IPython/config/default/ipcluster_config.py (+184/-0) IPython/config/default/ipcontroller_config.py (+136/-0) IPython/config/default/ipengine_config.py (+90/-0) IPython/config/default/ipython_config.py (+4/-4) IPython/config/loader.py (+9/-2) IPython/config/profile/ipython_config_cluster.py (+24/-0) IPython/core/application.py (+125/-61) IPython/core/builtin_trap.py (+1/-0) IPython/core/component.py (+24/-3) IPython/core/crashhandler.py (+1/-1) IPython/core/debugger.py (+76/-121) IPython/core/display_trap.py (+5/-5) IPython/core/embed.py (+2/-10) IPython/core/ipapi.py (+4/-1) IPython/core/ipapp.py (+70/-72) IPython/core/iplib.py (+43/-25) IPython/core/magic.py (+56/-29) IPython/core/oldusersetup.py (+0/-219) IPython/core/prefilter.py (+1/-1) IPython/core/tests/test_iplib.py (+1/-24) IPython/core/usage.py (+7/-7) IPython/extensions/parallelmagic.py (+181/-147) IPython/frontend/wx/ipythonx.py (+2/-3) IPython/gui/wx/wxIPython.py (+2/-2) IPython/kernel/asyncclient.py (+13/-12) IPython/kernel/client.py (+35/-43) IPython/kernel/clientconnector.py (+735/-112) IPython/kernel/clusterdir.py (+475/-0) IPython/kernel/config/__init__.py (+0/-126) IPython/kernel/configobjfactory.py (+79/-0) IPython/kernel/core/config/__init__.py (+0/-25) IPython/kernel/engineconnector.py (+79/-32) IPython/kernel/error.py (+8/-6) IPython/kernel/fcutil.py (+230/-26) IPython/kernel/ipclusterapp.py (+471/-0) IPython/kernel/ipcontrollerapp.py (+275/-0) IPython/kernel/ipengineapp.py (+248/-0) IPython/kernel/launcher.py (+869/-0) IPython/kernel/multiengine.py (+2/-3) IPython/kernel/multiengineclient.py (+11/-3) IPython/kernel/scripts/ipcluster (+10/-14) IPython/kernel/scripts/ipcluster.py (+0/-813) IPython/kernel/scripts/ipcontroller (+10/-12) IPython/kernel/scripts/ipcontroller.py (+0/-416) IPython/kernel/scripts/ipengine (+11/-11) IPython/kernel/scripts/ipengine.py (+0/-193) IPython/kernel/twistedutil.py (+34/-9) IPython/kernel/winhpcjob.py (+318/-0) IPython/lib/inputhookwx.py (+73/-62) IPython/lib/irunner.py (+3/-3) IPython/quarantine/InterpreterExec.py (+1/-1) IPython/quarantine/ipy_fsops.py (+2/-2) IPython/utils/genutils.py (+95/-86) IPython/utils/notification.py (+86/-68) IPython/utils/tests/test_notification.py (+72/-79) IPython/utils/traitlets.py (+34/-4) docs/Makefile (+2/-0) docs/autogen_api.py (+1/-0) docs/examples/core/new-embed.py (+17/-0) docs/examples/kernel/fractal.py (+90/-0) docs/examples/kernel/mcdriver.py (+54/-54) docs/examples/kernel/mcpricer.py (+42/-40) docs/examples/kernel/parallelpi.py (+54/-0) docs/examples/kernel/pidigits.py (+144/-0) docs/examples/kernel/wordfreq.py (+48/-5) docs/man/ipcluster.1 (+1/-1) docs/man/ipython.1 (+6/-6) docs/source/conf.py (+8/-5) docs/source/config/overview.txt (+4/-4) docs/source/development/index.txt (+1/-0) docs/source/development/ipgraph.txt (+59/-0) docs/source/interactive/reference.txt (+9/-9) docs/source/parallel/index.txt (+2/-0) docs/source/parallel/parallel_demos.txt (+282/-0) docs/source/parallel/parallel_winhpc.txt (+333/-0) docs/source/parallel/winhpc_index.txt (+14/-0) setup.py (+3/-3) setupbase.py (+2/-2) |
To merge this branch: | bzr merge lp:~ipython-dev/ipython/kernel-config |
Related bugs: |
Reviewer | Review Type | Date Requested | Status |
---|---|---|---|
Fernando Perez | Approve | ||
Vishal Vatsa | Pending | ||
Review via email: mp+14900@code.launchpad.net |
Commit message
Description of the change
Brian Granger (ellisonbg) wrote : | # |
- 1269. By Brian Granger
-
Added better documentation to command line programs.
- 1270. By Brian Granger
-
Initial draft of Windows HPC documentation.
- 1271. By Brian Granger
-
Work in the documentation.
- 1272. By bgranger <bgranger@red>
-
Adding figures for options pricing example.
- 1273. By Brian Granger
-
Final work on the Win HPC whitepaper.
Fernando Perez (fdo.perez) wrote : | # |
Summary: Excellent work! Many thanks for all of this.
I'm not merging it now just in case you want to do it yourself tomorrow (12/30) first thing or have some other little change you'd like to put in. But if you don't merge it I'll go ahead and do it, and will begin adding code I have (like testing stuff) so we can push towards 0.11. We still have cleanups and things to do, but all of that is best done on top of a more functional trunk, which this branch helps LOT with.
Per-commit comments:
-r1264: ok
-r1265: in kernel/
-r1266: ok, just upstream
-r1267: great, more examples!
-r1268: good, these are changes we'd discussed over the phone.
-r1269: better docs, excellent...
-r1270, 71, 72, 73: this document is stellar, we'd reviewed it already.
Brian Granger (ellisonbg) wrote : | # |
> I'm not merging it now just in case you want to do it yourself tomorrow (12/30) first thing or have some other little change you'd like to put in. But if you don't merge it I'll go ahead and do it, and will begin adding code I have (like testing stuff) so we can push towards 0.11. We still have cleanups and things to do, but all of that is best done on top of a more functional trunk, which this branch helps LOT with.
Thanks for the review. Go ahead and do the merge, I am working on
other things right now.
> Per-commit comments:
>
> -r1264: ok
>
> -r1265: in kernel/
Yes, I can add a message about this to the logs.
> -r1266: ok, just upstream
>
> -r1267: great, more examples!
>
> -r1268: good, these are changes we'd discussed over the phone.
>
> -r1269: better docs, excellent...
>
> -r1270, 71, 72, 73: this document is stellar, we'd reviewed it already.
>
> --
> https:/
> You proposed lp:~ipython-dev/ipython/kernel-config for merging.
>
--
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
<email address hidden>
<email address hidden>
Fernando Perez (fdo.perez) wrote : | # |
On Wed, Dec 30, 2009 at 8:39 AM, Brian Granger <email address hidden> wrote:
>
> Thanks for the review. Go ahead and do the merge, I am working on
> other things right now
Done, merged and pushed with an extra tiny fix. Thanks again, great job!
Preview Diff
1 | === removed file 'IPython/config/api.py' | |||
2 | --- IPython/config/api.py 2009-07-02 02:49:32 +0000 | |||
3 | +++ IPython/config/api.py 1970-01-01 00:00:00 +0000 | |||
4 | @@ -1,102 +0,0 @@ | |||
5 | 1 | # encoding: utf-8 | ||
6 | 2 | |||
7 | 3 | """This is the official entry point to IPython's configuration system. """ | ||
8 | 4 | |||
9 | 5 | __docformat__ = "restructuredtext en" | ||
10 | 6 | |||
11 | 7 | #------------------------------------------------------------------------------- | ||
12 | 8 | # Copyright (C) 2008 The IPython Development Team | ||
13 | 9 | # | ||
14 | 10 | # Distributed under the terms of the BSD License. The full license is in | ||
15 | 11 | # the file COPYING, distributed as part of this software. | ||
16 | 12 | #------------------------------------------------------------------------------- | ||
17 | 13 | |||
18 | 14 | #------------------------------------------------------------------------------- | ||
19 | 15 | # Imports | ||
20 | 16 | #------------------------------------------------------------------------------- | ||
21 | 17 | |||
22 | 18 | import os | ||
23 | 19 | from os.path import join as pjoin | ||
24 | 20 | |||
25 | 21 | from IPython.utils.genutils import get_home_dir, get_ipython_dir | ||
26 | 22 | from IPython.external.configobj import ConfigObj | ||
27 | 23 | |||
28 | 24 | |||
29 | 25 | class ConfigObjManager(object): | ||
30 | 26 | |||
31 | 27 | def __init__(self, configObj, filename): | ||
32 | 28 | self.current = configObj | ||
33 | 29 | self.current.indent_type = ' ' | ||
34 | 30 | self.filename = filename | ||
35 | 31 | # self.write_default_config_file() | ||
36 | 32 | |||
37 | 33 | def get_config_obj(self): | ||
38 | 34 | return self.current | ||
39 | 35 | |||
40 | 36 | def update_config_obj(self, newConfig): | ||
41 | 37 | self.current.merge(newConfig) | ||
42 | 38 | |||
43 | 39 | def update_config_obj_from_file(self, filename): | ||
44 | 40 | newConfig = ConfigObj(filename, file_error=False) | ||
45 | 41 | self.current.merge(newConfig) | ||
46 | 42 | |||
47 | 43 | def update_config_obj_from_default_file(self, ipythondir=None): | ||
48 | 44 | fname = self.resolve_file_path(self.filename, ipythondir) | ||
49 | 45 | self.update_config_obj_from_file(fname) | ||
50 | 46 | |||
51 | 47 | def write_config_obj_to_file(self, filename): | ||
52 | 48 | f = open(filename, 'w') | ||
53 | 49 | self.current.write(f) | ||
54 | 50 | f.close() | ||
55 | 51 | |||
56 | 52 | def write_default_config_file(self): | ||
57 | 53 | ipdir = get_ipython_dir() | ||
58 | 54 | fname = pjoin(ipdir, self.filename) | ||
59 | 55 | if not os.path.isfile(fname): | ||
60 | 56 | print "Writing the configuration file to: " + fname | ||
61 | 57 | self.write_config_obj_to_file(fname) | ||
62 | 58 | |||
63 | 59 | def _import(self, key): | ||
64 | 60 | package = '.'.join(key.split('.')[0:-1]) | ||
65 | 61 | obj = key.split('.')[-1] | ||
66 | 62 | execString = 'from %s import %s' % (package, obj) | ||
67 | 63 | exec execString | ||
68 | 64 | exec 'temp = %s' % obj | ||
69 | 65 | return temp | ||
70 | 66 | |||
71 | 67 | def resolve_file_path(self, filename, ipythondir = None): | ||
72 | 68 | """Resolve filenames into absolute paths. | ||
73 | 69 | |||
74 | 70 | This function looks in the following directories in order: | ||
75 | 71 | |||
76 | 72 | 1. In the current working directory or by absolute path with ~ expanded | ||
77 | 73 | 2. In ipythondir if that is set | ||
78 | 74 | 3. In the IPYTHONDIR environment variable if it exists | ||
79 | 75 | 4. In the ~/.ipython directory | ||
80 | 76 | |||
81 | 77 | Note: The IPYTHONDIR is also used by the trunk version of IPython so | ||
82 | 78 | changing it will also affect it was well. | ||
83 | 79 | """ | ||
84 | 80 | |||
85 | 81 | # In cwd or by absolute path with ~ expanded | ||
86 | 82 | trythis = os.path.expanduser(filename) | ||
87 | 83 | if os.path.isfile(trythis): | ||
88 | 84 | return trythis | ||
89 | 85 | |||
90 | 86 | # In ipythondir if it is set | ||
91 | 87 | if ipythondir is not None: | ||
92 | 88 | trythis = pjoin(ipythondir, filename) | ||
93 | 89 | if os.path.isfile(trythis): | ||
94 | 90 | return trythis | ||
95 | 91 | |||
96 | 92 | trythis = pjoin(get_ipython_dir(), filename) | ||
97 | 93 | if os.path.isfile(trythis): | ||
98 | 94 | return trythis | ||
99 | 95 | |||
100 | 96 | return None | ||
101 | 97 | |||
102 | 98 | |||
103 | 99 | |||
104 | 100 | |||
105 | 101 | |||
106 | 102 | |||
107 | 103 | 0 | ||
108 | === added file 'IPython/config/default/ipcluster_config.py' | |||
109 | --- IPython/config/default/ipcluster_config.py 1970-01-01 00:00:00 +0000 | |||
110 | +++ IPython/config/default/ipcluster_config.py 2009-11-21 00:11:10 +0000 | |||
111 | @@ -0,0 +1,184 @@ | |||
112 | 1 | import os | ||
113 | 2 | |||
114 | 3 | c = get_config() | ||
115 | 4 | |||
116 | 5 | #----------------------------------------------------------------------------- | ||
117 | 6 | # Select which launchers to use | ||
118 | 7 | #----------------------------------------------------------------------------- | ||
119 | 8 | |||
120 | 9 | # This allows you to control what method is used to start the controller | ||
121 | 10 | # and engines. The following methods are currently supported: | ||
122 | 11 | # - Start as a regular process on localhost. | ||
123 | 12 | # - Start using mpiexec. | ||
124 | 13 | # - Start using the Windows HPC Server 2008 scheduler | ||
125 | 14 | # - Start using PBS | ||
126 | 15 | # - Start using SSH (currently broken) | ||
127 | 16 | |||
128 | 17 | |||
129 | 18 | # The selected launchers can be configured below. | ||
130 | 19 | |||
131 | 20 | # Options are: | ||
132 | 21 | # - LocalControllerLauncher | ||
133 | 22 | # - MPIExecControllerLauncher | ||
134 | 23 | # - PBSControllerLauncher | ||
135 | 24 | # - WindowsHPCControllerLauncher | ||
136 | 25 | # c.Global.controller_launcher = 'IPython.kernel.launcher.LocalControllerLauncher' | ||
137 | 26 | |||
138 | 27 | # Options are: | ||
139 | 28 | # - LocalEngineSetLauncher | ||
140 | 29 | # - MPIExecEngineSetLauncher | ||
141 | 30 | # - PBSEngineSetLauncher | ||
142 | 31 | # - WindowsHPCEngineSetLauncher | ||
143 | 32 | # c.Global.engine_launcher = 'IPython.kernel.launcher.LocalEngineSetLauncher' | ||
144 | 33 | |||
145 | 34 | #----------------------------------------------------------------------------- | ||
146 | 35 | # Global configuration | ||
147 | 36 | #----------------------------------------------------------------------------- | ||
148 | 37 | |||
149 | 38 | # The default number of engines that will be started. This is overridden by | ||
150 | 39 | # the -n command line option: "ipcluster start -n 4" | ||
151 | 40 | # c.Global.n = 2 | ||
152 | 41 | |||
153 | 42 | # Log to a file in cluster_dir/log, otherwise just log to sys.stdout. | ||
154 | 43 | # c.Global.log_to_file = False | ||
155 | 44 | |||
156 | 45 | # Remove old logs from cluster_dir/log before starting. | ||
157 | 46 | # c.Global.clean_logs = True | ||
158 | 47 | |||
159 | 48 | # The working directory for the process. The application will use os.chdir | ||
160 | 49 | # to change to this directory before starting. | ||
161 | 50 | # c.Global.work_dir = os.getcwd() | ||
162 | 51 | |||
163 | 52 | |||
164 | 53 | #----------------------------------------------------------------------------- | ||
165 | 54 | # Local process launchers | ||
166 | 55 | #----------------------------------------------------------------------------- | ||
167 | 56 | |||
168 | 57 | # The command line arguments to call the controller with. | ||
169 | 58 | # c.LocalControllerLauncher.controller_args = \ | ||
170 | 59 | # ['--log-to-file','--log-level', '40'] | ||
171 | 60 | |||
172 | 61 | # The working directory for the controller | ||
173 | 62 | # c.LocalEngineSetLauncher.work_dir = u'' | ||
174 | 63 | |||
175 | 64 | # Command line argument passed to the engines. | ||
176 | 65 | # c.LocalEngineSetLauncher.engine_args = ['--log-to-file','--log-level', '40'] | ||
177 | 66 | |||
178 | 67 | #----------------------------------------------------------------------------- | ||
179 | 68 | # MPIExec launchers | ||
180 | 69 | #----------------------------------------------------------------------------- | ||
181 | 70 | |||
182 | 71 | # The mpiexec/mpirun command to use in started the controller. | ||
183 | 72 | # c.MPIExecControllerLauncher.mpi_cmd = ['mpiexec'] | ||
184 | 73 | |||
185 | 74 | # Additional arguments to pass to the actual mpiexec command. | ||
186 | 75 | # c.MPIExecControllerLauncher.mpi_args = [] | ||
187 | 76 | |||
188 | 77 | # The command line argument to call the controller with. | ||
189 | 78 | # c.MPIExecControllerLauncher.controller_args = \ | ||
190 | 79 | # ['--log-to-file','--log-level', '40'] | ||
191 | 80 | |||
192 | 81 | |||
193 | 82 | # The mpiexec/mpirun command to use in started the controller. | ||
194 | 83 | # c.MPIExecEngineSetLauncher.mpi_cmd = ['mpiexec'] | ||
195 | 84 | |||
196 | 85 | # Additional arguments to pass to the actual mpiexec command. | ||
197 | 86 | # c.MPIExecEngineSetLauncher.mpi_args = [] | ||
198 | 87 | |||
199 | 88 | # Command line argument passed to the engines. | ||
200 | 89 | # c.MPIExecEngineSetLauncher.engine_args = ['--log-to-file','--log-level', '40'] | ||
201 | 90 | |||
202 | 91 | # The default number of engines to start if not given elsewhere. | ||
203 | 92 | # c.MPIExecEngineSetLauncher.n = 1 | ||
204 | 93 | |||
205 | 94 | #----------------------------------------------------------------------------- | ||
206 | 95 | # SSH launchers | ||
207 | 96 | #----------------------------------------------------------------------------- | ||
208 | 97 | |||
209 | 98 | # Todo | ||
210 | 99 | |||
211 | 100 | |||
212 | 101 | #----------------------------------------------------------------------------- | ||
213 | 102 | # Unix batch (PBS) schedulers launchers | ||
214 | 103 | #----------------------------------------------------------------------------- | ||
215 | 104 | |||
216 | 105 | # The command line program to use to submit a PBS job. | ||
217 | 106 | # c.PBSControllerLauncher.submit_command = 'qsub' | ||
218 | 107 | |||
219 | 108 | # The command line program to use to delete a PBS job. | ||
220 | 109 | # c.PBSControllerLauncher.delete_command = 'qdel' | ||
221 | 110 | |||
222 | 111 | # A regular expression that takes the output of qsub and find the job id. | ||
223 | 112 | # c.PBSControllerLauncher.job_id_regexp = r'\d+' | ||
224 | 113 | |||
225 | 114 | # The batch submission script used to start the controller. This is where | ||
226 | 115 | # environment variables would be setup, etc. This string is interpolated using | ||
227 | 116 | # the Itpl module in IPython.external. Basically, you can use ${n} for the | ||
228 | 117 | # number of engine and ${cluster_dir} for the cluster_dir. | ||
229 | 118 | # c.PBSControllerLauncher.batch_template = """""" | ||
230 | 119 | |||
231 | 120 | # The name of the instantiated batch script that will actually be used to | ||
232 | 121 | # submit the job. This will be written to the cluster directory. | ||
233 | 122 | # c.PBSControllerLauncher.batch_file_name = u'pbs_batch_script_controller' | ||
234 | 123 | |||
235 | 124 | |||
236 | 125 | # The command line program to use to submit a PBS job. | ||
237 | 126 | # c.PBSEngineSetLauncher.submit_command = 'qsub' | ||
238 | 127 | |||
239 | 128 | # The command line program to use to delete a PBS job. | ||
240 | 129 | # c.PBSEngineSetLauncher.delete_command = 'qdel' | ||
241 | 130 | |||
242 | 131 | # A regular expression that takes the output of qsub and find the job id. | ||
243 | 132 | # c.PBSEngineSetLauncher.job_id_regexp = r'\d+' | ||
244 | 133 | |||
245 | 134 | # The batch submission script used to start the engines. This is where | ||
246 | 135 | # environment variables would be setup, etc. This string is interpolated using | ||
247 | 136 | # the Itpl module in IPython.external. Basically, you can use ${n} for the | ||
248 | 137 | # number of engine and ${cluster_dir} for the cluster_dir. | ||
249 | 138 | # c.PBSEngineSetLauncher.batch_template = """""" | ||
250 | 139 | |||
251 | 140 | # The name of the instantiated batch script that will actually be used to | ||
252 | 141 | # submit the job. This will be written to the cluster directory. | ||
253 | 142 | # c.PBSEngineSetLauncher.batch_file_name = u'pbs_batch_script_engines' | ||
254 | 143 | |||
255 | 144 | #----------------------------------------------------------------------------- | ||
256 | 145 | # Windows HPC Server 2008 launcher configuration | ||
257 | 146 | #----------------------------------------------------------------------------- | ||
258 | 147 | |||
259 | 148 | # c.IPControllerJob.job_name = 'IPController' | ||
260 | 149 | # c.IPControllerJob.is_exclusive = False | ||
261 | 150 | # c.IPControllerJob.username = r'USERDOMAIN\USERNAME' | ||
262 | 151 | # c.IPControllerJob.priority = 'Highest' | ||
263 | 152 | # c.IPControllerJob.requested_nodes = '' | ||
264 | 153 | # c.IPControllerJob.project = 'MyProject' | ||
265 | 154 | |||
266 | 155 | # c.IPControllerTask.task_name = 'IPController' | ||
267 | 156 | # c.IPControllerTask.controller_cmd = [u'ipcontroller.exe'] | ||
268 | 157 | # c.IPControllerTask.controller_args = ['--log-to-file', '--log-level', '40'] | ||
269 | 158 | # c.IPControllerTask.environment_variables = {} | ||
270 | 159 | |||
271 | 160 | # c.WindowsHPCControllerLauncher.scheduler = 'HEADNODE' | ||
272 | 161 | # c.WindowsHPCControllerLauncher.job_file_name = u'ipcontroller_job.xml' | ||
273 | 162 | |||
274 | 163 | |||
275 | 164 | # c.IPEngineSetJob.job_name = 'IPEngineSet' | ||
276 | 165 | # c.IPEngineSetJob.is_exclusive = False | ||
277 | 166 | # c.IPEngineSetJob.username = r'USERDOMAIN\USERNAME' | ||
278 | 167 | # c.IPEngineSetJob.priority = 'Highest' | ||
279 | 168 | # c.IPEngineSetJob.requested_nodes = '' | ||
280 | 169 | # c.IPEngineSetJob.project = 'MyProject' | ||
281 | 170 | |||
282 | 171 | # c.IPEngineTask.task_name = 'IPEngine' | ||
283 | 172 | # c.IPEngineTask.engine_cmd = [u'ipengine.exe'] | ||
284 | 173 | # c.IPEngineTask.engine_args = ['--log-to-file', '--log-level', '40'] | ||
285 | 174 | # c.IPEngineTask.environment_variables = {} | ||
286 | 175 | |||
287 | 176 | # c.WindowsHPCEngineSetLauncher.scheduler = 'HEADNODE' | ||
288 | 177 | # c.WindowsHPCEngineSetLauncher.job_file_name = u'ipengineset_job.xml' | ||
289 | 178 | |||
290 | 179 | |||
291 | 180 | |||
292 | 181 | |||
293 | 182 | |||
294 | 183 | |||
295 | 184 | |||
296 | 0 | 185 | ||
297 | === added file 'IPython/config/default/ipcontroller_config.py' | |||
298 | --- IPython/config/default/ipcontroller_config.py 1970-01-01 00:00:00 +0000 | |||
299 | +++ IPython/config/default/ipcontroller_config.py 2009-11-21 00:11:10 +0000 | |||
300 | @@ -0,0 +1,136 @@ | |||
301 | 1 | from IPython.config.loader import Config | ||
302 | 2 | |||
303 | 3 | c = get_config() | ||
304 | 4 | |||
305 | 5 | #----------------------------------------------------------------------------- | ||
306 | 6 | # Global configuration | ||
307 | 7 | #----------------------------------------------------------------------------- | ||
308 | 8 | |||
309 | 9 | # Basic Global config attributes | ||
310 | 10 | |||
311 | 11 | # Start up messages are logged to stdout using the logging module. | ||
312 | 12 | # These all happen before the twisted reactor is started and are | ||
313 | 13 | # useful for debugging purposes. Can be (10=DEBUG,20=INFO,30=WARN,40=CRITICAL) | ||
314 | 14 | # and smaller is more verbose. | ||
315 | 15 | # c.Global.log_level = 20 | ||
316 | 16 | |||
317 | 17 | # Log to a file in cluster_dir/log, otherwise just log to sys.stdout. | ||
318 | 18 | # c.Global.log_to_file = False | ||
319 | 19 | |||
320 | 20 | # Remove old logs from cluster_dir/log before starting. | ||
321 | 21 | # c.Global.clean_logs = True | ||
322 | 22 | |||
323 | 23 | # A list of Python statements that will be run before starting the | ||
324 | 24 | # controller. This is provided because occasionally certain things need to | ||
325 | 25 | # be imported in the controller for pickling to work. | ||
326 | 26 | # c.Global.import_statements = ['import math'] | ||
327 | 27 | |||
328 | 28 | # Reuse the controller's FURL files. If False, FURL files are regenerated | ||
329 | 29 | # each time the controller is run. If True, they will be reused, *but*, you | ||
330 | 30 | # also must set the network ports by hand. If set, this will override the | ||
331 | 31 | # values set for the client and engine connections below. | ||
332 | 32 | # c.Global.reuse_furls = True | ||
333 | 33 | |||
334 | 34 | # Enable SSL encryption on all connections to the controller. If set, this | ||
335 | 35 | # will override the values set for the client and engine connections below. | ||
336 | 36 | # c.Global.secure = True | ||
337 | 37 | |||
338 | 38 | # The working directory for the process. The application will use os.chdir | ||
339 | 39 | # to change to this directory before starting. | ||
340 | 40 | # c.Global.work_dir = os.getcwd() | ||
341 | 41 | |||
342 | 42 | #----------------------------------------------------------------------------- | ||
343 | 43 | # Configure the client services | ||
344 | 44 | #----------------------------------------------------------------------------- | ||
345 | 45 | |||
346 | 46 | # Basic client service config attributes | ||
347 | 47 | |||
348 | 48 | # The network interface the controller will listen on for client connections. | ||
349 | 49 | # This should be an IP address or hostname of the controller's host. The empty | ||
350 | 50 | # string means listen on all interfaces. | ||
351 | 51 | # c.FCClientServiceFactory.ip = '' | ||
352 | 52 | |||
353 | 53 | # The TCP/IP port the controller will listen on for client connections. If 0 | ||
354 | 54 | # a random port will be used. If the controller's host has a firewall running | ||
355 | 55 | # it must allow incoming traffic on this port. | ||
356 | 56 | # c.FCClientServiceFactory.port = 0 | ||
357 | 57 | |||
358 | 58 | # The client learns how to connect to the controller by looking at the | ||
359 | 59 | # location field embedded in the FURL. If this field is empty, all network | ||
360 | 60 | # interfaces that the controller is listening on will be listed. To have the | ||
361 | 61 | # client connect on a particular interface, list it here. | ||
362 | 62 | # c.FCClientServiceFactory.location = '' | ||
363 | 63 | |||
364 | 64 | # Use SSL encryption for the client connection. | ||
365 | 65 | # c.FCClientServiceFactory.secure = True | ||
366 | 66 | |||
367 | 67 | # Reuse the client FURL each time the controller is started. If set, you must | ||
368 | 68 | # also pick a specific network port above (FCClientServiceFactory.port). | ||
369 | 69 | # c.FCClientServiceFactory.reuse_furls = False | ||
370 | 70 | |||
371 | 71 | #----------------------------------------------------------------------------- | ||
372 | 72 | # Configure the engine services | ||
373 | 73 | #----------------------------------------------------------------------------- | ||
374 | 74 | |||
375 | 75 | # Basic config attributes for the engine services. | ||
376 | 76 | |||
377 | 77 | # The network interface the controller will listen on for engine connections. | ||
378 | 78 | # This should be an IP address or hostname of the controller's host. The empty | ||
379 | 79 | # string means listen on all interfaces. | ||
380 | 80 | # c.FCEngineServiceFactory.ip = '' | ||
381 | 81 | |||
382 | 82 | # The TCP/IP port the controller will listen on for engine connections. If 0 | ||
383 | 83 | # a random port will be used. If the controller's host has a firewall running | ||
384 | 84 | # it must allow incoming traffic on this port. | ||
385 | 85 | # c.FCEngineServiceFactory.port = 0 | ||
386 | 86 | |||
387 | 87 | # The engine learns how to connect to the controller by looking at the | ||
388 | 88 | # location field embedded in the FURL. If this field is empty, all network | ||
389 | 89 | # interfaces that the controller is listening on will be listed. To have the | ||
390 | 90 | # client connect on a particular interface, list it here. | ||
391 | 91 | # c.FCEngineServiceFactory.location = '' | ||
392 | 92 | |||
393 | 93 | # Use SSL encryption for the engine connection. | ||
394 | 94 | # c.FCEngineServiceFactory.secure = True | ||
395 | 95 | |||
396 | 96 | # Reuse the client FURL each time the controller is started. If set, you must | ||
397 | 97 | # also pick a specific network port above (FCClientServiceFactory.port). | ||
398 | 98 | # c.FCEngineServiceFactory.reuse_furls = False | ||
399 | 99 | |||
400 | 100 | #----------------------------------------------------------------------------- | ||
401 | 101 | # Developer level configuration attributes | ||
402 | 102 | #----------------------------------------------------------------------------- | ||
403 | 103 | |||
404 | 104 | # You shouldn't have to modify anything in this section. These attributes | ||
405 | 105 | # are more for developers who want to change the behavior of the controller | ||
406 | 106 | # at a fundamental level. | ||
407 | 107 | |||
408 | 108 | # c.FCClientServiceFactory.cert_file = u'ipcontroller-client.pem' | ||
409 | 109 | |||
410 | 110 | # default_client_interfaces = Config() | ||
411 | 111 | # default_client_interfaces.Task.interface_chain = [ | ||
412 | 112 | # 'IPython.kernel.task.ITaskController', | ||
413 | 113 | # 'IPython.kernel.taskfc.IFCTaskController' | ||
414 | 114 | # ] | ||
415 | 115 | # | ||
416 | 116 | # default_client_interfaces.Task.furl_file = u'ipcontroller-tc.furl' | ||
417 | 117 | # | ||
418 | 118 | # default_client_interfaces.MultiEngine.interface_chain = [ | ||
419 | 119 | # 'IPython.kernel.multiengine.IMultiEngine', | ||
420 | 120 | # 'IPython.kernel.multienginefc.IFCSynchronousMultiEngine' | ||
421 | 121 | # ] | ||
422 | 122 | # | ||
423 | 123 | # default_client_interfaces.MultiEngine.furl_file = u'ipcontroller-mec.furl' | ||
424 | 124 | # | ||
425 | 125 | # c.FCEngineServiceFactory.interfaces = default_client_interfaces | ||
426 | 126 | |||
427 | 127 | # c.FCEngineServiceFactory.cert_file = u'ipcontroller-engine.pem' | ||
428 | 128 | |||
429 | 129 | # default_engine_interfaces = Config() | ||
430 | 130 | # default_engine_interfaces.Default.interface_chain = [ | ||
431 | 131 | # 'IPython.kernel.enginefc.IFCControllerBase' | ||
432 | 132 | # ] | ||
433 | 133 | # | ||
434 | 134 | # default_engine_interfaces.Default.furl_file = u'ipcontroller-engine.furl' | ||
435 | 135 | # | ||
436 | 136 | # c.FCEngineServiceFactory.interfaces = default_engine_interfaces | ||
437 | 0 | 137 | ||
438 | === added file 'IPython/config/default/ipengine_config.py' | |||
439 | --- IPython/config/default/ipengine_config.py 1970-01-01 00:00:00 +0000 | |||
440 | +++ IPython/config/default/ipengine_config.py 2009-11-21 00:11:10 +0000 | |||
441 | @@ -0,0 +1,90 @@ | |||
442 | 1 | c = get_config() | ||
443 | 2 | |||
444 | 3 | #----------------------------------------------------------------------------- | ||
445 | 4 | # Global configuration | ||
446 | 5 | #----------------------------------------------------------------------------- | ||
447 | 6 | |||
448 | 7 | # Start up messages are logged to stdout using the logging module. | ||
449 | 8 | # These all happen before the twisted reactor is started and are | ||
450 | 9 | # useful for debugging purposes. Can be (10=DEBUG,20=INFO,30=WARN,40=CRITICAL) | ||
451 | 10 | # and smaller is more verbose. | ||
452 | 11 | # c.Global.log_level = 20 | ||
453 | 12 | |||
454 | 13 | # Log to a file in cluster_dir/log, otherwise just log to sys.stdout. | ||
455 | 14 | # c.Global.log_to_file = False | ||
456 | 15 | |||
457 | 16 | # Remove old logs from cluster_dir/log before starting. | ||
458 | 17 | # c.Global.clean_logs = True | ||
459 | 18 | |||
460 | 19 | # A list of strings that will be executed in the users namespace on the engine | ||
461 | 20 | # before it connects to the controller. | ||
462 | 21 | # c.Global.exec_lines = ['import numpy'] | ||
463 | 22 | |||
464 | 23 | # The engine will try to connect to the controller multiple times, to allow | ||
465 | 24 | # the controller time to startup and write its FURL file. These parameters | ||
466 | 25 | # control the number of retries (connect_max_tries) and the initial delay | ||
467 | 26 | # (connect_delay) between attemps. The actual delay between attempts gets | ||
468 | 27 | # longer each time by a factor of 1.5 (delay[i] = 1.5*delay[i-1]) | ||
469 | 28 | # those attemps. | ||
470 | 29 | # c.Global.connect_delay = 0.1 | ||
471 | 30 | # c.Global.connect_max_tries = 15 | ||
472 | 31 | |||
473 | 32 | # By default, the engine will look for the controller's FURL file in its own | ||
474 | 33 | # cluster directory. Sometimes, the FURL file will be elsewhere and this | ||
475 | 34 | # attribute can be set to the full path of the FURL file. | ||
476 | 35 | # c.Global.furl_file = u'' | ||
477 | 36 | |||
478 | 37 | # The working directory for the process. The application will use os.chdir | ||
479 | 38 | # to change to this directory before starting. | ||
480 | 39 | # c.Global.work_dir = os.getcwd() | ||
481 | 40 | |||
482 | 41 | #----------------------------------------------------------------------------- | ||
483 | 42 | # MPI configuration | ||
484 | 43 | #----------------------------------------------------------------------------- | ||
485 | 44 | |||
486 | 45 | # Upon starting the engine can be configured to call MPI_Init. This section | ||
487 | 46 | # configures that. | ||
488 | 47 | |||
489 | 48 | # Select which MPI section to execute to setup MPI. The value of this | ||
490 | 49 | # attribute must match the name of another attribute in the MPI config | ||
491 | 50 | # section (mpi4py, pytrilinos, etc.). This can also be set by the --mpi | ||
492 | 51 | # command line option. | ||
493 | 52 | # c.MPI.use = '' | ||
494 | 53 | |||
495 | 54 | # Initialize MPI using mpi4py. To use this, set c.MPI.use = 'mpi4py' to use | ||
496 | 55 | # --mpi=mpi4py at the command line. | ||
497 | 56 | # c.MPI.mpi4py = """from mpi4py import MPI as mpi | ||
498 | 57 | # mpi.size = mpi.COMM_WORLD.Get_size() | ||
499 | 58 | # mpi.rank = mpi.COMM_WORLD.Get_rank() | ||
500 | 59 | # """ | ||
501 | 60 | |||
502 | 61 | # Initialize MPI using pytrilinos. To use this, set c.MPI.use = 'pytrilinos' | ||
503 | 62 | # to use --mpi=pytrilinos at the command line. | ||
504 | 63 | # c.MPI.pytrilinos = """from PyTrilinos import Epetra | ||
505 | 64 | # class SimpleStruct: | ||
506 | 65 | # pass | ||
507 | 66 | # mpi = SimpleStruct() | ||
508 | 67 | # mpi.rank = 0 | ||
509 | 68 | # mpi.size = 0 | ||
510 | 69 | # """ | ||
511 | 70 | |||
512 | 71 | #----------------------------------------------------------------------------- | ||
513 | 72 | # Developer level configuration attributes | ||
514 | 73 | #----------------------------------------------------------------------------- | ||
515 | 74 | |||
516 | 75 | # You shouldn't have to modify anything in this section. These attributes | ||
517 | 76 | # are more for developers who want to change the behavior of the controller | ||
518 | 77 | # at a fundamental level. | ||
519 | 78 | |||
520 | 79 | # You should not have to change these attributes. | ||
521 | 80 | |||
522 | 81 | # c.Global.shell_class = 'IPython.kernel.core.interpreter.Interpreter' | ||
523 | 82 | |||
524 | 83 | # c.Global.furl_file_name = u'ipcontroller-engine.furl' | ||
525 | 84 | |||
526 | 85 | |||
527 | 86 | |||
528 | 87 | |||
529 | 88 | |||
530 | 89 | |||
531 | 90 | |||
532 | 0 | 91 | ||
533 | === modified file 'IPython/config/default/ipython_config.py' | |||
534 | --- IPython/config/default/ipython_config.py 2009-09-28 05:57:44 +0000 | |||
535 | +++ IPython/config/default/ipython_config.py 2009-11-21 00:11:10 +0000 | |||
536 | @@ -13,7 +13,7 @@ | |||
537 | 13 | 13 | ||
538 | 14 | # Set this to determine the detail of what is logged at startup. | 14 | # Set this to determine the detail of what is logged at startup. |
539 | 15 | # The default is 30 and possible values are 0,10,20,30,40,50. | 15 | # The default is 30 and possible values are 0,10,20,30,40,50. |
541 | 16 | c.Global.log_level = 20 | 16 | # c.Global.log_level = 20 |
542 | 17 | 17 | ||
543 | 18 | # This should be a list of importable Python modules that have an | 18 | # This should be a list of importable Python modules that have an |
544 | 19 | # load_in_ipython(ip) method. This method gets called when the extension | 19 | # load_in_ipython(ip) method. This method gets called when the extension |
545 | @@ -35,7 +35,7 @@ | |||
546 | 35 | # These files are run in IPython in the user's namespace. Files with a .py | 35 | # These files are run in IPython in the user's namespace. Files with a .py |
547 | 36 | # extension need to be pure Python. Files with a .ipy extension can have | 36 | # extension need to be pure Python. Files with a .ipy extension can have |
548 | 37 | # custom IPython syntax (like magics, etc.). | 37 | # custom IPython syntax (like magics, etc.). |
550 | 38 | # These files need to be in the cwd, the ipythondir or be absolute paths. | 38 | # These files need to be in the cwd, the ipython_dir or be absolute paths. |
551 | 39 | # c.Global.exec_files = [ | 39 | # c.Global.exec_files = [ |
552 | 40 | # 'mycode.py', | 40 | # 'mycode.py', |
553 | 41 | # 'fancy.ipy' | 41 | # 'fancy.ipy' |
554 | @@ -71,9 +71,9 @@ | |||
555 | 71 | 71 | ||
556 | 72 | # c.InteractiveShell.logstart = True | 72 | # c.InteractiveShell.logstart = True |
557 | 73 | 73 | ||
559 | 74 | # c.InteractiveShell.logfile = 'ipython_log.py' | 74 | # c.InteractiveShell.logfile = u'ipython_log.py' |
560 | 75 | 75 | ||
562 | 76 | # c.InteractiveShell.logappend = 'mylog.py' | 76 | # c.InteractiveShell.logappend = u'mylog.py' |
563 | 77 | 77 | ||
564 | 78 | # c.InteractiveShell.object_info_string_level = 0 | 78 | # c.InteractiveShell.object_info_string_level = 0 |
565 | 79 | 79 | ||
566 | 80 | 80 | ||
567 | === modified file 'IPython/config/loader.py' | |||
568 | --- IPython/config/loader.py 2009-09-15 05:54:31 +0000 | |||
569 | +++ IPython/config/loader.py 2009-11-21 00:11:10 +0000 | |||
570 | @@ -244,8 +244,14 @@ | |||
571 | 244 | # with the parents. | 244 | # with the parents. |
572 | 245 | def load_subconfig(fname): | 245 | def load_subconfig(fname): |
573 | 246 | loader = PyFileConfigLoader(fname, self.path) | 246 | loader = PyFileConfigLoader(fname, self.path) |
576 | 247 | sub_config = loader.load_config() | 247 | try: |
577 | 248 | self.config._merge(sub_config) | 248 | sub_config = loader.load_config() |
578 | 249 | except IOError: | ||
579 | 250 | # Pass silently if the sub config is not there. This happens | ||
580 | 251 | # when a user us using a profile, but not the default config. | ||
581 | 252 | pass | ||
582 | 253 | else: | ||
583 | 254 | self.config._merge(sub_config) | ||
584 | 249 | 255 | ||
585 | 250 | # Again, this needs to be a closure and should be used in config | 256 | # Again, this needs to be a closure and should be used in config |
586 | 251 | # files to get the config being loaded. | 257 | # files to get the config being loaded. |
587 | @@ -271,6 +277,7 @@ | |||
588 | 271 | class NoConfigDefault(object): pass | 277 | class NoConfigDefault(object): pass |
589 | 272 | NoConfigDefault = NoConfigDefault() | 278 | NoConfigDefault = NoConfigDefault() |
590 | 273 | 279 | ||
591 | 280 | |||
592 | 274 | class ArgParseConfigLoader(CommandLineConfigLoader): | 281 | class ArgParseConfigLoader(CommandLineConfigLoader): |
593 | 275 | 282 | ||
594 | 276 | # arguments = [(('-f','--file'),dict(type=str,dest='file'))] | 283 | # arguments = [(('-f','--file'),dict(type=str,dest='file'))] |
595 | 277 | 284 | ||
596 | === renamed file 'IPython/config/profile/__init_.py' => 'IPython/config/profile/__init__.py' | |||
597 | === added file 'IPython/config/profile/ipython_config_cluster.py' | |||
598 | --- IPython/config/profile/ipython_config_cluster.py 1970-01-01 00:00:00 +0000 | |||
599 | +++ IPython/config/profile/ipython_config_cluster.py 2009-11-21 00:11:10 +0000 | |||
600 | @@ -0,0 +1,24 @@ | |||
601 | 1 | c = get_config() | ||
602 | 2 | |||
603 | 3 | # This can be used at any point in a config file to load a sub config | ||
604 | 4 | # and merge it into the current one. | ||
605 | 5 | load_subconfig('ipython_config.py') | ||
606 | 6 | |||
607 | 7 | lines = """ | ||
608 | 8 | from IPython.kernel.client import * | ||
609 | 9 | """ | ||
610 | 10 | |||
611 | 11 | # You have to make sure that attributes that are containers already | ||
612 | 12 | # exist before using them. Simple assigning a new list will override | ||
613 | 13 | # all previous values. | ||
614 | 14 | if hasattr(c.Global, 'exec_lines'): | ||
615 | 15 | c.Global.exec_lines.append(lines) | ||
616 | 16 | else: | ||
617 | 17 | c.Global.exec_lines = [lines] | ||
618 | 18 | |||
619 | 19 | # Load the parallelmagic extension to enable %result, %px, %autopx magics. | ||
620 | 20 | if hasattr(c.Global, 'extensions'): | ||
621 | 21 | c.Global.extensions.append('parallelmagic') | ||
622 | 22 | else: | ||
623 | 23 | c.Global.extensions = ['parallelmagic'] | ||
624 | 24 | |||
625 | 0 | 25 | ||
626 | === modified file 'IPython/core/application.py' | |||
627 | --- IPython/core/application.py 2009-09-18 03:26:43 +0000 | |||
628 | +++ IPython/core/application.py 2009-11-21 00:11:10 +0000 | |||
629 | @@ -1,7 +1,13 @@ | |||
630 | 1 | #!/usr/bin/env python | 1 | #!/usr/bin/env python |
631 | 2 | # encoding: utf-8 | 2 | # encoding: utf-8 |
632 | 3 | """ | 3 | """ |
634 | 4 | An application for IPython | 4 | An application for IPython. |
635 | 5 | |||
636 | 6 | All top-level applications should use the classes in this module for | ||
637 | 7 | handling configuration and creating componenets. | ||
638 | 8 | |||
639 | 9 | The job of an :class:`Application` is to create the master configuration | ||
640 | 10 | object and then create the components, passing the config to them. | ||
641 | 5 | 11 | ||
642 | 6 | Authors: | 12 | Authors: |
643 | 7 | 13 | ||
644 | @@ -26,10 +32,9 @@ | |||
645 | 26 | import logging | 32 | import logging |
646 | 27 | import os | 33 | import os |
647 | 28 | import sys | 34 | import sys |
648 | 29 | import traceback | ||
649 | 30 | from copy import deepcopy | ||
650 | 31 | 35 | ||
652 | 32 | from IPython.utils.genutils import get_ipython_dir, filefind | 36 | from IPython.core import release |
653 | 37 | from IPython.utils.genutils import get_ipython_dir | ||
654 | 33 | from IPython.config.loader import ( | 38 | from IPython.config.loader import ( |
655 | 34 | PyFileConfigLoader, | 39 | PyFileConfigLoader, |
656 | 35 | ArgParseConfigLoader, | 40 | ArgParseConfigLoader, |
657 | @@ -42,22 +47,27 @@ | |||
658 | 42 | #----------------------------------------------------------------------------- | 47 | #----------------------------------------------------------------------------- |
659 | 43 | 48 | ||
660 | 44 | 49 | ||
662 | 45 | class IPythonArgParseConfigLoader(ArgParseConfigLoader): | 50 | class BaseAppArgParseConfigLoader(ArgParseConfigLoader): |
663 | 46 | """Default command line options for IPython based applications.""" | 51 | """Default command line options for IPython based applications.""" |
664 | 47 | 52 | ||
665 | 48 | def _add_other_arguments(self): | 53 | def _add_other_arguments(self): |
668 | 49 | self.parser.add_argument('-ipythondir',dest='Global.ipythondir',type=str, | 54 | self.parser.add_argument('--ipython-dir', |
669 | 50 | help='Set to override default location of Global.ipythondir.', | 55 | dest='Global.ipython_dir',type=unicode, |
670 | 56 | help='Set to override default location of Global.ipython_dir.', | ||
671 | 51 | default=NoConfigDefault, | 57 | default=NoConfigDefault, |
674 | 52 | metavar='Global.ipythondir') | 58 | metavar='Global.ipython_dir') |
675 | 53 | self.parser.add_argument('-p','-profile',dest='Global.profile',type=str, | 59 | self.parser.add_argument('-p', '--profile', |
676 | 60 | dest='Global.profile',type=unicode, | ||
677 | 54 | help='The string name of the ipython profile to be used.', | 61 | help='The string name of the ipython profile to be used.', |
678 | 55 | default=NoConfigDefault, | 62 | default=NoConfigDefault, |
679 | 56 | metavar='Global.profile') | 63 | metavar='Global.profile') |
681 | 57 | self.parser.add_argument('-log_level',dest="Global.log_level",type=int, | 64 | self.parser.add_argument('--log-level', |
682 | 65 | dest="Global.log_level",type=int, | ||
683 | 58 | help='Set the log level (0,10,20,30,40,50). Default is 30.', | 66 | help='Set the log level (0,10,20,30,40,50). Default is 30.', |
686 | 59 | default=NoConfigDefault) | 67 | default=NoConfigDefault, |
687 | 60 | self.parser.add_argument('-config_file',dest='Global.config_file',type=str, | 68 | metavar='Global.log_level') |
688 | 69 | self.parser.add_argument('--config-file', | ||
689 | 70 | dest='Global.config_file',type=unicode, | ||
690 | 61 | help='Set the config file name to override default.', | 71 | help='Set the config file name to override default.', |
691 | 62 | default=NoConfigDefault, | 72 | default=NoConfigDefault, |
692 | 63 | metavar='Global.config_file') | 73 | metavar='Global.config_file') |
693 | @@ -68,20 +78,24 @@ | |||
694 | 68 | 78 | ||
695 | 69 | 79 | ||
696 | 70 | class Application(object): | 80 | class Application(object): |
699 | 71 | """Load a config, construct an app and run it. | 81 | """Load a config, construct components and set them running.""" |
698 | 72 | """ | ||
700 | 73 | 82 | ||
703 | 74 | config_file_name = 'ipython_config.py' | 83 | name = u'ipython' |
704 | 75 | name = 'ipython' | 84 | description = 'IPython: an enhanced interactive Python shell.' |
705 | 85 | config_file_name = u'ipython_config.py' | ||
706 | 86 | default_log_level = logging.WARN | ||
707 | 76 | 87 | ||
708 | 77 | def __init__(self): | 88 | def __init__(self): |
709 | 89 | self._exiting = False | ||
710 | 78 | self.init_logger() | 90 | self.init_logger() |
711 | 91 | # Track the default and actual separately because some messages are | ||
712 | 92 | # only printed if we aren't using the default. | ||
713 | 79 | self.default_config_file_name = self.config_file_name | 93 | self.default_config_file_name = self.config_file_name |
714 | 80 | 94 | ||
715 | 81 | def init_logger(self): | 95 | def init_logger(self): |
716 | 82 | self.log = logging.getLogger(self.__class__.__name__) | 96 | self.log = logging.getLogger(self.__class__.__name__) |
717 | 83 | # This is used as the default until the command line arguments are read. | 97 | # This is used as the default until the command line arguments are read. |
719 | 84 | self.log.setLevel(logging.WARN) | 98 | self.log.setLevel(self.default_log_level) |
720 | 85 | self._log_handler = logging.StreamHandler() | 99 | self._log_handler = logging.StreamHandler() |
721 | 86 | self._log_formatter = logging.Formatter("[%(name)s] %(message)s") | 100 | self._log_formatter = logging.Formatter("[%(name)s] %(message)s") |
722 | 87 | self._log_handler.setFormatter(self._log_formatter) | 101 | self._log_handler.setFormatter(self._log_formatter) |
723 | @@ -98,16 +112,24 @@ | |||
724 | 98 | def start(self): | 112 | def start(self): |
725 | 99 | """Start the application.""" | 113 | """Start the application.""" |
726 | 100 | self.attempt(self.create_default_config) | 114 | self.attempt(self.create_default_config) |
727 | 115 | self.log_default_config() | ||
728 | 116 | self.set_default_config_log_level() | ||
729 | 101 | self.attempt(self.pre_load_command_line_config) | 117 | self.attempt(self.pre_load_command_line_config) |
730 | 102 | self.attempt(self.load_command_line_config, action='abort') | 118 | self.attempt(self.load_command_line_config, action='abort') |
731 | 119 | self.set_command_line_config_log_level() | ||
732 | 103 | self.attempt(self.post_load_command_line_config) | 120 | self.attempt(self.post_load_command_line_config) |
734 | 104 | self.attempt(self.find_ipythondir) | 121 | self.log_command_line_config() |
735 | 122 | self.attempt(self.find_ipython_dir) | ||
736 | 123 | self.attempt(self.find_resources) | ||
737 | 105 | self.attempt(self.find_config_file_name) | 124 | self.attempt(self.find_config_file_name) |
738 | 106 | self.attempt(self.find_config_file_paths) | 125 | self.attempt(self.find_config_file_paths) |
739 | 107 | self.attempt(self.pre_load_file_config) | 126 | self.attempt(self.pre_load_file_config) |
740 | 108 | self.attempt(self.load_file_config) | 127 | self.attempt(self.load_file_config) |
741 | 128 | self.set_file_config_log_level() | ||
742 | 109 | self.attempt(self.post_load_file_config) | 129 | self.attempt(self.post_load_file_config) |
743 | 130 | self.log_file_config() | ||
744 | 110 | self.attempt(self.merge_configs) | 131 | self.attempt(self.merge_configs) |
745 | 132 | self.log_master_config() | ||
746 | 111 | self.attempt(self.pre_construct) | 133 | self.attempt(self.pre_construct) |
747 | 112 | self.attempt(self.construct) | 134 | self.attempt(self.construct) |
748 | 113 | self.attempt(self.post_construct) | 135 | self.attempt(self.post_construct) |
749 | @@ -127,65 +149,89 @@ | |||
750 | 127 | don't belong to a particular component. | 149 | don't belong to a particular component. |
751 | 128 | """ | 150 | """ |
752 | 129 | self.default_config = Config() | 151 | self.default_config = Config() |
754 | 130 | self.default_config.Global.ipythondir = get_ipython_dir() | 152 | self.default_config.Global.ipython_dir = get_ipython_dir() |
755 | 153 | self.default_config.Global.log_level = self.log_level | ||
756 | 154 | |||
757 | 155 | def log_default_config(self): | ||
758 | 131 | self.log.debug('Default config loaded:') | 156 | self.log.debug('Default config loaded:') |
759 | 132 | self.log.debug(repr(self.default_config)) | 157 | self.log.debug(repr(self.default_config)) |
760 | 133 | 158 | ||
761 | 159 | def set_default_config_log_level(self): | ||
762 | 160 | try: | ||
763 | 161 | self.log_level = self.default_config.Global.log_level | ||
764 | 162 | except AttributeError: | ||
765 | 163 | # Fallback to the default_log_level class attribute | ||
766 | 164 | pass | ||
767 | 165 | |||
768 | 134 | def create_command_line_config(self): | 166 | def create_command_line_config(self): |
769 | 135 | """Create and return a command line config loader.""" | 167 | """Create and return a command line config loader.""" |
771 | 136 | return IPythonArgParseConfigLoader(description=self.name) | 168 | return BaseAppArgParseConfigLoader( |
772 | 169 | description=self.description, | ||
773 | 170 | version=release.version | ||
774 | 171 | ) | ||
775 | 137 | 172 | ||
776 | 138 | def pre_load_command_line_config(self): | 173 | def pre_load_command_line_config(self): |
777 | 139 | """Do actions just before loading the command line config.""" | 174 | """Do actions just before loading the command line config.""" |
778 | 140 | pass | 175 | pass |
779 | 141 | 176 | ||
780 | 142 | def load_command_line_config(self): | 177 | def load_command_line_config(self): |
786 | 143 | """Load the command line config. | 178 | """Load the command line config.""" |
782 | 144 | |||
783 | 145 | This method also sets ``self.debug``. | ||
784 | 146 | """ | ||
785 | 147 | |||
787 | 148 | loader = self.create_command_line_config() | 179 | loader = self.create_command_line_config() |
788 | 149 | self.command_line_config = loader.load_config() | 180 | self.command_line_config = loader.load_config() |
789 | 150 | self.extra_args = loader.get_extra_args() | 181 | self.extra_args = loader.get_extra_args() |
790 | 151 | 182 | ||
791 | 183 | def set_command_line_config_log_level(self): | ||
792 | 152 | try: | 184 | try: |
793 | 153 | self.log_level = self.command_line_config.Global.log_level | 185 | self.log_level = self.command_line_config.Global.log_level |
794 | 154 | except AttributeError: | 186 | except AttributeError: |
796 | 155 | pass # Use existing value which is set in Application.init_logger. | 187 | pass |
797 | 188 | |||
798 | 189 | def post_load_command_line_config(self): | ||
799 | 190 | """Do actions just after loading the command line config.""" | ||
800 | 191 | pass | ||
801 | 192 | |||
802 | 193 | def log_command_line_config(self): | ||
803 | 156 | self.log.debug("Command line config loaded:") | 194 | self.log.debug("Command line config loaded:") |
804 | 157 | self.log.debug(repr(self.command_line_config)) | 195 | self.log.debug(repr(self.command_line_config)) |
805 | 158 | 196 | ||
811 | 159 | def post_load_command_line_config(self): | 197 | def find_ipython_dir(self): |
807 | 160 | """Do actions just after loading the command line config.""" | ||
808 | 161 | pass | ||
809 | 162 | |||
810 | 163 | def find_ipythondir(self): | ||
812 | 164 | """Set the IPython directory. | 198 | """Set the IPython directory. |
813 | 165 | 199 | ||
815 | 166 | This sets ``self.ipythondir``, but the actual value that is passed | 200 | This sets ``self.ipython_dir``, but the actual value that is passed |
816 | 167 | to the application is kept in either ``self.default_config`` or | 201 | to the application is kept in either ``self.default_config`` or |
818 | 168 | ``self.command_line_config``. This also added ``self.ipythondir`` to | 202 | ``self.command_line_config``. This also adds ``self.ipython_dir`` to |
819 | 169 | ``sys.path`` so config files there can be references by other config | 203 | ``sys.path`` so config files there can be references by other config |
820 | 170 | files. | 204 | files. |
821 | 171 | """ | 205 | """ |
822 | 172 | 206 | ||
823 | 173 | try: | 207 | try: |
825 | 174 | self.ipythondir = self.command_line_config.Global.ipythondir | 208 | self.ipython_dir = self.command_line_config.Global.ipython_dir |
826 | 175 | except AttributeError: | 209 | except AttributeError: |
832 | 176 | self.ipythondir = self.default_config.Global.ipythondir | 210 | self.ipython_dir = self.default_config.Global.ipython_dir |
833 | 177 | sys.path.append(os.path.abspath(self.ipythondir)) | 211 | sys.path.append(os.path.abspath(self.ipython_dir)) |
834 | 178 | if not os.path.isdir(self.ipythondir): | 212 | if not os.path.isdir(self.ipython_dir): |
835 | 179 | os.makedirs(self.ipythondir, mode = 0777) | 213 | os.makedirs(self.ipython_dir, mode=0777) |
836 | 180 | self.log.debug("IPYTHONDIR set to: %s" % self.ipythondir) | 214 | self.log.debug("IPYTHON_DIR set to: %s" % self.ipython_dir) |
837 | 215 | |||
838 | 216 | def find_resources(self): | ||
839 | 217 | """Find other resources that need to be in place. | ||
840 | 218 | |||
841 | 219 | Things like cluster directories need to be in place to find the | ||
842 | 220 | config file. These happen right after the IPython directory has | ||
843 | 221 | been set. | ||
844 | 222 | """ | ||
845 | 223 | pass | ||
846 | 181 | 224 | ||
847 | 182 | def find_config_file_name(self): | 225 | def find_config_file_name(self): |
848 | 183 | """Find the config file name for this application. | 226 | """Find the config file name for this application. |
849 | 184 | 227 | ||
850 | 228 | This must set ``self.config_file_name`` to the filename of the | ||
851 | 229 | config file to use (just the filename). The search paths for the | ||
852 | 230 | config file are set in :meth:`find_config_file_paths` and then passed | ||
853 | 231 | to the config file loader where they are resolved to an absolute path. | ||
854 | 232 | |||
855 | 185 | If a profile has been set at the command line, this will resolve | 233 | If a profile has been set at the command line, this will resolve |
859 | 186 | it. The search paths for the config file are set in | 234 | it. |
857 | 187 | :meth:`find_config_file_paths` and then passed to the config file | ||
858 | 188 | loader where they are resolved to an absolute path. | ||
860 | 189 | """ | 235 | """ |
861 | 190 | 236 | ||
862 | 191 | try: | 237 | try: |
863 | @@ -196,14 +242,18 @@ | |||
864 | 196 | try: | 242 | try: |
865 | 197 | self.profile_name = self.command_line_config.Global.profile | 243 | self.profile_name = self.command_line_config.Global.profile |
866 | 198 | name_parts = self.config_file_name.split('.') | 244 | name_parts = self.config_file_name.split('.') |
868 | 199 | name_parts.insert(1, '_' + self.profile_name + '.') | 245 | name_parts.insert(1, u'_' + self.profile_name + u'.') |
869 | 200 | self.config_file_name = ''.join(name_parts) | 246 | self.config_file_name = ''.join(name_parts) |
870 | 201 | except AttributeError: | 247 | except AttributeError: |
871 | 202 | pass | 248 | pass |
872 | 203 | 249 | ||
873 | 204 | def find_config_file_paths(self): | 250 | def find_config_file_paths(self): |
876 | 205 | """Set the search paths for resolving the config file.""" | 251 | """Set the search paths for resolving the config file. |
877 | 206 | self.config_file_paths = (os.getcwd(), self.ipythondir) | 252 | |
878 | 253 | This must set ``self.config_file_paths`` to a sequence of search | ||
879 | 254 | paths to pass to the config file loader. | ||
880 | 255 | """ | ||
881 | 256 | self.config_file_paths = (os.getcwd(), self.ipython_dir) | ||
882 | 207 | 257 | ||
883 | 208 | def pre_load_file_config(self): | 258 | def pre_load_file_config(self): |
884 | 209 | """Do actions before the config file is loaded.""" | 259 | """Do actions before the config file is loaded.""" |
885 | @@ -216,7 +266,7 @@ | |||
886 | 216 | ``CONFIG_FILE`` config variable is set to the resolved config file | 266 | ``CONFIG_FILE`` config variable is set to the resolved config file |
887 | 217 | location. If not successful, an empty config is used. | 267 | location. If not successful, an empty config is used. |
888 | 218 | """ | 268 | """ |
890 | 219 | self.log.debug("Attempting to load config file: <%s>" % self.config_file_name) | 269 | self.log.debug("Attempting to load config file: %s" % self.config_file_name) |
891 | 220 | loader = PyFileConfigLoader(self.config_file_name, | 270 | loader = PyFileConfigLoader(self.config_file_name, |
892 | 221 | path=self.config_file_paths) | 271 | path=self.config_file_paths) |
893 | 222 | try: | 272 | try: |
894 | @@ -225,19 +275,18 @@ | |||
895 | 225 | except IOError: | 275 | except IOError: |
896 | 226 | # Only warn if the default config file was NOT being used. | 276 | # Only warn if the default config file was NOT being used. |
897 | 227 | if not self.config_file_name==self.default_config_file_name: | 277 | if not self.config_file_name==self.default_config_file_name: |
899 | 228 | self.log.warn("Config file not found, skipping: <%s>" % \ | 278 | self.log.warn("Config file not found, skipping: %s" % \ |
900 | 229 | self.config_file_name, exc_info=True) | 279 | self.config_file_name, exc_info=True) |
901 | 230 | self.file_config = Config() | 280 | self.file_config = Config() |
902 | 231 | except: | 281 | except: |
905 | 232 | self.log.warn("Error loading config file: <%s>" % \ | 282 | self.log.warn("Error loading config file: %s" % \ |
906 | 233 | self.config_file_name, exc_info=True) | 283 | self.config_file_name, exc_info=True) |
907 | 234 | self.file_config = Config() | 284 | self.file_config = Config() |
911 | 235 | else: | 285 | |
912 | 236 | self.log.debug("Config file loaded: <%s>" % loader.full_filename) | 286 | def set_file_config_log_level(self): |
910 | 237 | self.log.debug(repr(self.file_config)) | ||
913 | 238 | # We need to keeep self.log_level updated. But we only use the value | 287 | # We need to keeep self.log_level updated. But we only use the value |
914 | 239 | # of the file_config if a value was not specified at the command | 288 | # of the file_config if a value was not specified at the command |
916 | 240 | # line. | 289 | # line, because the command line overrides everything. |
917 | 241 | if not hasattr(self.command_line_config.Global, 'log_level'): | 290 | if not hasattr(self.command_line_config.Global, 'log_level'): |
918 | 242 | try: | 291 | try: |
919 | 243 | self.log_level = self.file_config.Global.log_level | 292 | self.log_level = self.file_config.Global.log_level |
920 | @@ -248,6 +297,11 @@ | |||
921 | 248 | """Do actions after the config file is loaded.""" | 297 | """Do actions after the config file is loaded.""" |
922 | 249 | pass | 298 | pass |
923 | 250 | 299 | ||
924 | 300 | def log_file_config(self): | ||
925 | 301 | if hasattr(self.file_config.Global, 'config_file'): | ||
926 | 302 | self.log.debug("Config file loaded: %s" % self.file_config.Global.config_file) | ||
927 | 303 | self.log.debug(repr(self.file_config)) | ||
928 | 304 | |||
929 | 251 | def merge_configs(self): | 305 | def merge_configs(self): |
930 | 252 | """Merge the default, command line and file config objects.""" | 306 | """Merge the default, command line and file config objects.""" |
931 | 253 | config = Config() | 307 | config = Config() |
932 | @@ -255,6 +309,8 @@ | |||
933 | 255 | config._merge(self.file_config) | 309 | config._merge(self.file_config) |
934 | 256 | config._merge(self.command_line_config) | 310 | config._merge(self.command_line_config) |
935 | 257 | self.master_config = config | 311 | self.master_config = config |
936 | 312 | |||
937 | 313 | def log_master_config(self): | ||
938 | 258 | self.log.debug("Master config created:") | 314 | self.log.debug("Master config created:") |
939 | 259 | self.log.debug(repr(self.master_config)) | 315 | self.log.debug(repr(self.master_config)) |
940 | 260 | 316 | ||
941 | @@ -280,21 +336,29 @@ | |||
942 | 280 | 336 | ||
943 | 281 | def abort(self): | 337 | def abort(self): |
944 | 282 | """Abort the starting of the application.""" | 338 | """Abort the starting of the application.""" |
947 | 283 | self.log.critical("Aborting application: %s" % self.name, exc_info=True) | 339 | if self._exiting: |
948 | 284 | sys.exit(1) | 340 | pass |
949 | 341 | else: | ||
950 | 342 | self.log.critical("Aborting application: %s" % self.name, exc_info=True) | ||
951 | 343 | self._exiting = True | ||
952 | 344 | sys.exit(1) | ||
953 | 285 | 345 | ||
957 | 286 | def exit(self): | 346 | def exit(self, exit_status=0): |
958 | 287 | self.log.critical("Aborting application: %s" % self.name) | 347 | if self._exiting: |
959 | 288 | sys.exit(1) | 348 | pass |
960 | 349 | else: | ||
961 | 350 | self.log.debug("Exiting application: %s" % self.name) | ||
962 | 351 | self._exiting = True | ||
963 | 352 | sys.exit(exit_status) | ||
964 | 289 | 353 | ||
965 | 290 | def attempt(self, func, action='abort'): | 354 | def attempt(self, func, action='abort'): |
966 | 291 | try: | 355 | try: |
967 | 292 | func() | 356 | func() |
968 | 293 | except SystemExit: | 357 | except SystemExit: |
970 | 294 | self.exit() | 358 | raise |
971 | 295 | except: | 359 | except: |
972 | 296 | if action == 'abort': | 360 | if action == 'abort': |
973 | 297 | self.abort() | 361 | self.abort() |
974 | 298 | elif action == 'exit': | 362 | elif action == 'exit': |
975 | 299 | self.exit() | ||
976 | 300 | |||
977 | 301 | \ No newline at end of file | 363 | \ No newline at end of file |
978 | 364 | self.exit(0) | ||
979 | 365 | |||
980 | 302 | 366 | ||
981 | === modified file 'IPython/core/builtin_trap.py' | |||
982 | --- IPython/core/builtin_trap.py 2009-10-09 22:54:40 +0000 | |||
983 | +++ IPython/core/builtin_trap.py 2009-11-21 00:11:10 +0000 | |||
984 | @@ -86,6 +86,7 @@ | |||
985 | 86 | """Store ipython references in the __builtin__ namespace.""" | 86 | """Store ipython references in the __builtin__ namespace.""" |
986 | 87 | self.add_builtin('exit', Quitter(self.shell, 'exit')) | 87 | self.add_builtin('exit', Quitter(self.shell, 'exit')) |
987 | 88 | self.add_builtin('quit', Quitter(self.shell, 'quit')) | 88 | self.add_builtin('quit', Quitter(self.shell, 'quit')) |
988 | 89 | self.add_builtin('get_ipython', self.shell.get_ipython) | ||
989 | 89 | 90 | ||
990 | 90 | # Recursive reload function | 91 | # Recursive reload function |
991 | 91 | try: | 92 | try: |
992 | 92 | 93 | ||
993 | === modified file 'IPython/core/component.py' | |||
994 | --- IPython/core/component.py 2009-09-28 05:57:44 +0000 | |||
995 | +++ IPython/core/component.py 2009-11-21 00:11:11 +0000 | |||
996 | @@ -237,14 +237,20 @@ | |||
997 | 237 | self.config = config | 237 | self.config = config |
998 | 238 | # We used to deepcopy, but for now we are trying to just save | 238 | # We used to deepcopy, but for now we are trying to just save |
999 | 239 | # by reference. This *could* have side effects as all components | 239 | # by reference. This *could* have side effects as all components |
1001 | 240 | # will share config. | 240 | # will share config. In fact, I did find such a side effect in |
1002 | 241 | # _config_changed below. If a config attribute value was a mutable type | ||
1003 | 242 | # all instances of a component were getting the same copy, effectively | ||
1004 | 243 | # making that a class attribute. | ||
1005 | 241 | # self.config = deepcopy(config) | 244 | # self.config = deepcopy(config) |
1006 | 242 | else: | 245 | else: |
1007 | 243 | if self.parent is not None: | 246 | if self.parent is not None: |
1008 | 244 | self.config = self.parent.config | 247 | self.config = self.parent.config |
1009 | 245 | # We used to deepcopy, but for now we are trying to just save | 248 | # We used to deepcopy, but for now we are trying to just save |
1010 | 246 | # by reference. This *could* have side effects as all components | 249 | # by reference. This *could* have side effects as all components |
1012 | 247 | # will share config. | 250 | # will share config. In fact, I did find such a side effect in |
1013 | 251 | # _config_changed below. If a config attribute value was a mutable type | ||
1014 | 252 | # all instances of a component were getting the same copy, effectively | ||
1015 | 253 | # making that a class attribute. | ||
1016 | 248 | # self.config = deepcopy(self.parent.config) | 254 | # self.config = deepcopy(self.parent.config) |
1017 | 249 | 255 | ||
1018 | 250 | self.created = datetime.datetime.now() | 256 | self.created = datetime.datetime.now() |
1019 | @@ -296,14 +302,29 @@ | |||
1020 | 296 | if new._has_section(sname): | 302 | if new._has_section(sname): |
1021 | 297 | my_config = new[sname] | 303 | my_config = new[sname] |
1022 | 298 | for k, v in traitlets.items(): | 304 | for k, v in traitlets.items(): |
1023 | 305 | # Don't allow traitlets with config=True to start with | ||
1024 | 306 | # uppercase. Otherwise, they are confused with Config | ||
1025 | 307 | # subsections. But, developers shouldn't have uppercase | ||
1026 | 308 | # attributes anyways! (PEP 6) | ||
1027 | 309 | if k[0].upper()==k[0] and not k.startswith('_'): | ||
1028 | 310 | raise ComponentError('Component traitlets with ' | ||
1029 | 311 | 'config=True must start with a lowercase so they are ' | ||
1030 | 312 | 'not confused with Config subsections: %s.%s' % \ | ||
1031 | 313 | (self.__class__.__name__, k)) | ||
1032 | 299 | try: | 314 | try: |
1033 | 315 | # Here we grab the value from the config | ||
1034 | 316 | # If k has the naming convention of a config | ||
1035 | 317 | # section, it will be auto created. | ||
1036 | 300 | config_value = my_config[k] | 318 | config_value = my_config[k] |
1037 | 301 | except KeyError: | 319 | except KeyError: |
1038 | 302 | pass | 320 | pass |
1039 | 303 | else: | 321 | else: |
1040 | 304 | # print "Setting %s.%s from %s.%s=%r" % \ | 322 | # print "Setting %s.%s from %s.%s=%r" % \ |
1041 | 305 | # (self.__class__.__name__,k,sname,k,config_value) | 323 | # (self.__class__.__name__,k,sname,k,config_value) |
1043 | 306 | setattr(self, k, config_value) | 324 | # We have to do a deepcopy here if we don't deepcopy the entire |
1044 | 325 | # config object. If we don't, a mutable config_value will be | ||
1045 | 326 | # shared by all instances, effectively making it a class attribute. | ||
1046 | 327 | setattr(self, k, deepcopy(config_value)) | ||
1047 | 307 | 328 | ||
1048 | 308 | @property | 329 | @property |
1049 | 309 | def children(self): | 330 | def children(self): |
1050 | 310 | 331 | ||
1051 | === modified file 'IPython/core/crashhandler.py' | |||
1052 | --- IPython/core/crashhandler.py 2009-08-26 20:54:07 +0000 | |||
1053 | +++ IPython/core/crashhandler.py 2009-11-21 00:11:11 +0000 | |||
1054 | @@ -124,7 +124,7 @@ | |||
1055 | 124 | #color_scheme = 'Linux' # dbg | 124 | #color_scheme = 'Linux' # dbg |
1056 | 125 | 125 | ||
1057 | 126 | try: | 126 | try: |
1059 | 127 | rptdir = self.IP.config.IPYTHONDIR | 127 | rptdir = self.IP.ipython_dir |
1060 | 128 | except: | 128 | except: |
1061 | 129 | rptdir = os.getcwd() | 129 | rptdir = os.getcwd() |
1062 | 130 | if not os.path.isdir(rptdir): | 130 | if not os.path.isdir(rptdir): |
1063 | 131 | 131 | ||
1064 | === modified file 'IPython/core/debugger.py' | |||
1065 | --- IPython/core/debugger.py 2009-08-19 21:56:41 +0000 | |||
1066 | +++ IPython/core/debugger.py 2009-11-21 00:11:11 +0000 | |||
1067 | @@ -70,6 +70,7 @@ | |||
1068 | 70 | def BdbQuit_IPython_excepthook(self,et,ev,tb): | 70 | def BdbQuit_IPython_excepthook(self,et,ev,tb): |
1069 | 71 | print 'Exiting Debugger.' | 71 | print 'Exiting Debugger.' |
1070 | 72 | 72 | ||
1071 | 73 | |||
1072 | 73 | class Tracer(object): | 74 | class Tracer(object): |
1073 | 74 | """Class for local debugging, similar to pdb.set_trace. | 75 | """Class for local debugging, similar to pdb.set_trace. |
1074 | 75 | 76 | ||
1075 | @@ -105,12 +106,10 @@ | |||
1076 | 105 | from the Python standard library for usage details. | 106 | from the Python standard library for usage details. |
1077 | 106 | """ | 107 | """ |
1078 | 107 | 108 | ||
1079 | 108 | global __IPYTHON__ | ||
1080 | 109 | try: | 109 | try: |
1083 | 110 | __IPYTHON__ | 110 | ip = ipapi.get() |
1084 | 111 | except NameError: | 111 | except: |
1085 | 112 | # Outside of ipython, we set our own exception hook manually | 112 | # Outside of ipython, we set our own exception hook manually |
1086 | 113 | __IPYTHON__ = ipapi.get() | ||
1087 | 114 | BdbQuit_excepthook.excepthook_ori = sys.excepthook | 113 | BdbQuit_excepthook.excepthook_ori = sys.excepthook |
1088 | 115 | sys.excepthook = BdbQuit_excepthook | 114 | sys.excepthook = BdbQuit_excepthook |
1089 | 116 | def_colors = 'NoColor' | 115 | def_colors = 'NoColor' |
1090 | @@ -122,9 +121,8 @@ | |||
1091 | 122 | pass | 121 | pass |
1092 | 123 | else: | 122 | else: |
1093 | 124 | # In ipython, we use its custom exception handler mechanism | 123 | # In ipython, we use its custom exception handler mechanism |
1094 | 125 | ip = ipapi.get() | ||
1095 | 126 | def_colors = ip.colors | 124 | def_colors = ip.colors |
1097 | 127 | ip.set_custom_exc((bdb.BdbQuit,),BdbQuit_IPython_excepthook) | 125 | ip.set_custom_exc((bdb.BdbQuit,), BdbQuit_IPython_excepthook) |
1098 | 128 | 126 | ||
1099 | 129 | if colors is None: | 127 | if colors is None: |
1100 | 130 | colors = def_colors | 128 | colors = def_colors |
1101 | @@ -138,6 +136,7 @@ | |||
1102 | 138 | 136 | ||
1103 | 139 | self.debugger.set_trace(sys._getframe().f_back) | 137 | self.debugger.set_trace(sys._getframe().f_back) |
1104 | 140 | 138 | ||
1105 | 139 | |||
1106 | 141 | def decorate_fn_with_doc(new_fn, old_fn, additional_text=""): | 140 | def decorate_fn_with_doc(new_fn, old_fn, additional_text=""): |
1107 | 142 | """Make new_fn have old_fn's doc string. This is particularly useful | 141 | """Make new_fn have old_fn's doc string. This is particularly useful |
1108 | 143 | for the do_... commands that hook into the help system. | 142 | for the do_... commands that hook into the help system. |
1109 | @@ -149,6 +148,7 @@ | |||
1110 | 149 | wrapper.__doc__ = old_fn.__doc__ + additional_text | 148 | wrapper.__doc__ = old_fn.__doc__ + additional_text |
1111 | 150 | return wrapper | 149 | return wrapper |
1112 | 151 | 150 | ||
1113 | 151 | |||
1114 | 152 | def _file_lines(fname): | 152 | def _file_lines(fname): |
1115 | 153 | """Return the contents of a named file as a list of lines. | 153 | """Return the contents of a named file as a list of lines. |
1116 | 154 | 154 | ||
1117 | @@ -164,143 +164,98 @@ | |||
1118 | 164 | outfile.close() | 164 | outfile.close() |
1119 | 165 | return out | 165 | return out |
1120 | 166 | 166 | ||
1121 | 167 | |||
1122 | 167 | class Pdb(OldPdb): | 168 | class Pdb(OldPdb): |
1123 | 168 | """Modified Pdb class, does not load readline.""" | 169 | """Modified Pdb class, does not load readline.""" |
1124 | 169 | 170 | ||
1128 | 170 | if sys.version[:3] >= '2.5' or has_pydb: | 171 | def __init__(self,color_scheme='NoColor',completekey=None, |
1129 | 171 | def __init__(self,color_scheme='NoColor',completekey=None, | 172 | stdin=None, stdout=None): |
1127 | 172 | stdin=None, stdout=None): | ||
1130 | 173 | 173 | ||
1138 | 174 | # Parent constructor: | 174 | # Parent constructor: |
1139 | 175 | if has_pydb and completekey is None: | 175 | if has_pydb and completekey is None: |
1140 | 176 | OldPdb.__init__(self,stdin=stdin,stdout=Term.cout) | 176 | OldPdb.__init__(self,stdin=stdin,stdout=Term.cout) |
1141 | 177 | else: | 177 | else: |
1142 | 178 | OldPdb.__init__(self,completekey,stdin,stdout) | 178 | OldPdb.__init__(self,completekey,stdin,stdout) |
1136 | 179 | |||
1137 | 180 | self.prompt = prompt # The default prompt is '(Pdb)' | ||
1143 | 181 | 179 | ||
1238 | 182 | # IPython changes... | 180 | self.prompt = prompt # The default prompt is '(Pdb)' |
1145 | 183 | self.is_pydb = has_pydb | ||
1146 | 184 | |||
1147 | 185 | if self.is_pydb: | ||
1148 | 186 | |||
1149 | 187 | # iplib.py's ipalias seems to want pdb's checkline | ||
1150 | 188 | # which located in pydb.fn | ||
1151 | 189 | import pydb.fns | ||
1152 | 190 | self.checkline = lambda filename, lineno: \ | ||
1153 | 191 | pydb.fns.checkline(self, filename, lineno) | ||
1154 | 192 | |||
1155 | 193 | self.curframe = None | ||
1156 | 194 | self.do_restart = self.new_do_restart | ||
1157 | 195 | |||
1158 | 196 | self.old_all_completions = __IPYTHON__.Completer.all_completions | ||
1159 | 197 | __IPYTHON__.Completer.all_completions=self.all_completions | ||
1160 | 198 | |||
1161 | 199 | self.do_list = decorate_fn_with_doc(self.list_command_pydb, | ||
1162 | 200 | OldPdb.do_list) | ||
1163 | 201 | self.do_l = self.do_list | ||
1164 | 202 | self.do_frame = decorate_fn_with_doc(self.new_do_frame, | ||
1165 | 203 | OldPdb.do_frame) | ||
1166 | 204 | |||
1167 | 205 | self.aliases = {} | ||
1168 | 206 | |||
1169 | 207 | # Create color table: we copy the default one from the traceback | ||
1170 | 208 | # module and add a few attributes needed for debugging | ||
1171 | 209 | self.color_scheme_table = exception_colors() | ||
1172 | 210 | |||
1173 | 211 | # shorthands | ||
1174 | 212 | C = coloransi.TermColors | ||
1175 | 213 | cst = self.color_scheme_table | ||
1176 | 214 | |||
1177 | 215 | cst['NoColor'].colors.breakpoint_enabled = C.NoColor | ||
1178 | 216 | cst['NoColor'].colors.breakpoint_disabled = C.NoColor | ||
1179 | 217 | |||
1180 | 218 | cst['Linux'].colors.breakpoint_enabled = C.LightRed | ||
1181 | 219 | cst['Linux'].colors.breakpoint_disabled = C.Red | ||
1182 | 220 | |||
1183 | 221 | cst['LightBG'].colors.breakpoint_enabled = C.LightRed | ||
1184 | 222 | cst['LightBG'].colors.breakpoint_disabled = C.Red | ||
1185 | 223 | |||
1186 | 224 | self.set_colors(color_scheme) | ||
1187 | 225 | |||
1188 | 226 | # Add a python parser so we can syntax highlight source while | ||
1189 | 227 | # debugging. | ||
1190 | 228 | self.parser = PyColorize.Parser() | ||
1191 | 229 | |||
1192 | 230 | |||
1193 | 231 | else: | ||
1194 | 232 | # Ugly hack: for Python 2.3-2.4, we can't call the parent constructor, | ||
1195 | 233 | # because it binds readline and breaks tab-completion. This means we | ||
1196 | 234 | # have to COPY the constructor here. | ||
1197 | 235 | def __init__(self,color_scheme='NoColor'): | ||
1198 | 236 | bdb.Bdb.__init__(self) | ||
1199 | 237 | cmd.Cmd.__init__(self,completekey=None) # don't load readline | ||
1200 | 238 | self.prompt = 'ipdb> ' # The default prompt is '(Pdb)' | ||
1201 | 239 | self.aliases = {} | ||
1202 | 240 | |||
1203 | 241 | # These two lines are part of the py2.4 constructor, let's put them | ||
1204 | 242 | # unconditionally here as they won't cause any problems in 2.3. | ||
1205 | 243 | self.mainpyfile = '' | ||
1206 | 244 | self._wait_for_mainpyfile = 0 | ||
1207 | 245 | |||
1208 | 246 | # Read $HOME/.pdbrc and ./.pdbrc | ||
1209 | 247 | try: | ||
1210 | 248 | self.rcLines = _file_lines(os.path.join(os.environ['HOME'], | ||
1211 | 249 | ".pdbrc")) | ||
1212 | 250 | except KeyError: | ||
1213 | 251 | self.rcLines = [] | ||
1214 | 252 | self.rcLines.extend(_file_lines(".pdbrc")) | ||
1215 | 253 | |||
1216 | 254 | # Create color table: we copy the default one from the traceback | ||
1217 | 255 | # module and add a few attributes needed for debugging | ||
1218 | 256 | self.color_scheme_table = exception_colors() | ||
1219 | 257 | |||
1220 | 258 | # shorthands | ||
1221 | 259 | C = coloransi.TermColors | ||
1222 | 260 | cst = self.color_scheme_table | ||
1223 | 261 | |||
1224 | 262 | cst['NoColor'].colors.breakpoint_enabled = C.NoColor | ||
1225 | 263 | cst['NoColor'].colors.breakpoint_disabled = C.NoColor | ||
1226 | 264 | |||
1227 | 265 | cst['Linux'].colors.breakpoint_enabled = C.LightRed | ||
1228 | 266 | cst['Linux'].colors.breakpoint_disabled = C.Red | ||
1229 | 267 | |||
1230 | 268 | cst['LightBG'].colors.breakpoint_enabled = C.LightRed | ||
1231 | 269 | cst['LightBG'].colors.breakpoint_disabled = C.Red | ||
1232 | 270 | |||
1233 | 271 | self.set_colors(color_scheme) | ||
1234 | 272 | |||
1235 | 273 | # Add a python parser so we can syntax highlight source while | ||
1236 | 274 | # debugging. | ||
1237 | 275 | self.parser = PyColorize.Parser() | ||
1239 | 276 | 181 | ||
1240 | 182 | # IPython changes... | ||
1241 | 183 | self.is_pydb = has_pydb | ||
1242 | 184 | |||
1243 | 185 | self.shell = ipapi.get() | ||
1244 | 186 | |||
1245 | 187 | if self.is_pydb: | ||
1246 | 188 | |||
1247 | 189 | # iplib.py's ipalias seems to want pdb's checkline | ||
1248 | 190 | # which located in pydb.fn | ||
1249 | 191 | import pydb.fns | ||
1250 | 192 | self.checkline = lambda filename, lineno: \ | ||
1251 | 193 | pydb.fns.checkline(self, filename, lineno) | ||
1252 | 194 | |||
1253 | 195 | self.curframe = None | ||
1254 | 196 | self.do_restart = self.new_do_restart | ||
1255 | 197 | |||
1256 | 198 | self.old_all_completions = self.shell.Completer.all_completions | ||
1257 | 199 | self.shell.Completer.all_completions=self.all_completions | ||
1258 | 200 | |||
1259 | 201 | self.do_list = decorate_fn_with_doc(self.list_command_pydb, | ||
1260 | 202 | OldPdb.do_list) | ||
1261 | 203 | self.do_l = self.do_list | ||
1262 | 204 | self.do_frame = decorate_fn_with_doc(self.new_do_frame, | ||
1263 | 205 | OldPdb.do_frame) | ||
1264 | 206 | |||
1265 | 207 | self.aliases = {} | ||
1266 | 208 | |||
1267 | 209 | # Create color table: we copy the default one from the traceback | ||
1268 | 210 | # module and add a few attributes needed for debugging | ||
1269 | 211 | self.color_scheme_table = exception_colors() | ||
1270 | 212 | |||
1271 | 213 | # shorthands | ||
1272 | 214 | C = coloransi.TermColors | ||
1273 | 215 | cst = self.color_scheme_table | ||
1274 | 216 | |||
1275 | 217 | cst['NoColor'].colors.breakpoint_enabled = C.NoColor | ||
1276 | 218 | cst['NoColor'].colors.breakpoint_disabled = C.NoColor | ||
1277 | 219 | |||
1278 | 220 | cst['Linux'].colors.breakpoint_enabled = C.LightRed | ||
1279 | 221 | cst['Linux'].colors.breakpoint_disabled = C.Red | ||
1280 | 222 | |||
1281 | 223 | cst['LightBG'].colors.breakpoint_enabled = C.LightRed | ||
1282 | 224 | cst['LightBG'].colors.breakpoint_disabled = C.Red | ||
1283 | 225 | |||
1284 | 226 | self.set_colors(color_scheme) | ||
1285 | 227 | |||
1286 | 228 | # Add a python parser so we can syntax highlight source while | ||
1287 | 229 | # debugging. | ||
1288 | 230 | self.parser = PyColorize.Parser() | ||
1289 | 231 | |||
1290 | 277 | def set_colors(self, scheme): | 232 | def set_colors(self, scheme): |
1291 | 278 | """Shorthand access to the color table scheme selector method.""" | 233 | """Shorthand access to the color table scheme selector method.""" |
1292 | 279 | self.color_scheme_table.set_active_scheme(scheme) | 234 | self.color_scheme_table.set_active_scheme(scheme) |
1293 | 280 | 235 | ||
1294 | 281 | def interaction(self, frame, traceback): | 236 | def interaction(self, frame, traceback): |
1296 | 282 | __IPYTHON__.set_completer_frame(frame) | 237 | self.shell.set_completer_frame(frame) |
1297 | 283 | OldPdb.interaction(self, frame, traceback) | 238 | OldPdb.interaction(self, frame, traceback) |
1298 | 284 | 239 | ||
1299 | 285 | def new_do_up(self, arg): | 240 | def new_do_up(self, arg): |
1300 | 286 | OldPdb.do_up(self, arg) | 241 | OldPdb.do_up(self, arg) |
1302 | 287 | __IPYTHON__.set_completer_frame(self.curframe) | 242 | self.shell.set_completer_frame(self.curframe) |
1303 | 288 | do_u = do_up = decorate_fn_with_doc(new_do_up, OldPdb.do_up) | 243 | do_u = do_up = decorate_fn_with_doc(new_do_up, OldPdb.do_up) |
1304 | 289 | 244 | ||
1305 | 290 | def new_do_down(self, arg): | 245 | def new_do_down(self, arg): |
1306 | 291 | OldPdb.do_down(self, arg) | 246 | OldPdb.do_down(self, arg) |
1308 | 292 | __IPYTHON__.set_completer_frame(self.curframe) | 247 | self.shell.set_completer_frame(self.curframe) |
1309 | 293 | 248 | ||
1310 | 294 | do_d = do_down = decorate_fn_with_doc(new_do_down, OldPdb.do_down) | 249 | do_d = do_down = decorate_fn_with_doc(new_do_down, OldPdb.do_down) |
1311 | 295 | 250 | ||
1312 | 296 | def new_do_frame(self, arg): | 251 | def new_do_frame(self, arg): |
1313 | 297 | OldPdb.do_frame(self, arg) | 252 | OldPdb.do_frame(self, arg) |
1315 | 298 | __IPYTHON__.set_completer_frame(self.curframe) | 253 | self.shell.set_completer_frame(self.curframe) |
1316 | 299 | 254 | ||
1317 | 300 | def new_do_quit(self, arg): | 255 | def new_do_quit(self, arg): |
1318 | 301 | 256 | ||
1319 | 302 | if hasattr(self, 'old_all_completions'): | 257 | if hasattr(self, 'old_all_completions'): |
1321 | 303 | __IPYTHON__.Completer.all_completions=self.old_all_completions | 258 | self.shell.Completer.all_completions=self.old_all_completions |
1322 | 304 | 259 | ||
1323 | 305 | 260 | ||
1324 | 306 | return OldPdb.do_quit(self, arg) | 261 | return OldPdb.do_quit(self, arg) |
1325 | @@ -314,7 +269,7 @@ | |||
1326 | 314 | return self.do_quit(arg) | 269 | return self.do_quit(arg) |
1327 | 315 | 270 | ||
1328 | 316 | def postloop(self): | 271 | def postloop(self): |
1330 | 317 | __IPYTHON__.set_completer_frame(None) | 272 | self.shell.set_completer_frame(None) |
1331 | 318 | 273 | ||
1332 | 319 | def print_stack_trace(self): | 274 | def print_stack_trace(self): |
1333 | 320 | try: | 275 | try: |
1334 | @@ -331,7 +286,7 @@ | |||
1335 | 331 | # vds: >> | 286 | # vds: >> |
1336 | 332 | frame, lineno = frame_lineno | 287 | frame, lineno = frame_lineno |
1337 | 333 | filename = frame.f_code.co_filename | 288 | filename = frame.f_code.co_filename |
1339 | 334 | __IPYTHON__.hooks.synchronize_with_editor(filename, lineno, 0) | 289 | self.shell.hooks.synchronize_with_editor(filename, lineno, 0) |
1340 | 335 | # vds: << | 290 | # vds: << |
1341 | 336 | 291 | ||
1342 | 337 | def format_stack_entry(self, frame_lineno, lprefix=': ', context = 3): | 292 | def format_stack_entry(self, frame_lineno, lprefix=': ', context = 3): |
1343 | @@ -500,7 +455,7 @@ | |||
1344 | 500 | # vds: >> | 455 | # vds: >> |
1345 | 501 | lineno = first | 456 | lineno = first |
1346 | 502 | filename = self.curframe.f_code.co_filename | 457 | filename = self.curframe.f_code.co_filename |
1348 | 503 | __IPYTHON__.hooks.synchronize_with_editor(filename, lineno, 0) | 458 | self.shell.hooks.synchronize_with_editor(filename, lineno, 0) |
1349 | 504 | # vds: << | 459 | # vds: << |
1350 | 505 | 460 | ||
1351 | 506 | do_l = do_list | 461 | do_l = do_list |
1352 | @@ -509,16 +464,16 @@ | |||
1353 | 509 | """The debugger interface to magic_pdef""" | 464 | """The debugger interface to magic_pdef""" |
1354 | 510 | namespaces = [('Locals', self.curframe.f_locals), | 465 | namespaces = [('Locals', self.curframe.f_locals), |
1355 | 511 | ('Globals', self.curframe.f_globals)] | 466 | ('Globals', self.curframe.f_globals)] |
1357 | 512 | __IPYTHON__.magic_pdef(arg, namespaces=namespaces) | 467 | self.shell.magic_pdef(arg, namespaces=namespaces) |
1358 | 513 | 468 | ||
1359 | 514 | def do_pdoc(self, arg): | 469 | def do_pdoc(self, arg): |
1360 | 515 | """The debugger interface to magic_pdoc""" | 470 | """The debugger interface to magic_pdoc""" |
1361 | 516 | namespaces = [('Locals', self.curframe.f_locals), | 471 | namespaces = [('Locals', self.curframe.f_locals), |
1362 | 517 | ('Globals', self.curframe.f_globals)] | 472 | ('Globals', self.curframe.f_globals)] |
1364 | 518 | __IPYTHON__.magic_pdoc(arg, namespaces=namespaces) | 473 | self.shell.magic_pdoc(arg, namespaces=namespaces) |
1365 | 519 | 474 | ||
1366 | 520 | def do_pinfo(self, arg): | 475 | def do_pinfo(self, arg): |
1367 | 521 | """The debugger equivalant of ?obj""" | 476 | """The debugger equivalant of ?obj""" |
1368 | 522 | namespaces = [('Locals', self.curframe.f_locals), | 477 | namespaces = [('Locals', self.curframe.f_locals), |
1369 | 523 | ('Globals', self.curframe.f_globals)] | 478 | ('Globals', self.curframe.f_globals)] |
1371 | 524 | __IPYTHON__.magic_pinfo("pinfo %s" % arg, namespaces=namespaces) | 479 | self.shell.magic_pinfo("pinfo %s" % arg, namespaces=namespaces) |
1372 | 525 | 480 | ||
1373 | === modified file 'IPython/core/display_trap.py' | |||
1374 | --- IPython/core/display_trap.py 2009-10-09 22:54:40 +0000 | |||
1375 | +++ IPython/core/display_trap.py 2009-11-21 00:11:11 +0000 | |||
1376 | @@ -46,11 +46,11 @@ | |||
1377 | 46 | # Only turn off the trap when the outermost call to __exit__ is made. | 46 | # Only turn off the trap when the outermost call to __exit__ is made. |
1378 | 47 | self._nested_level = 0 | 47 | self._nested_level = 0 |
1379 | 48 | 48 | ||
1385 | 49 | @auto_attr | 49 | # @auto_attr |
1386 | 50 | def shell(self): | 50 | # def shell(self): |
1387 | 51 | return Component.get_instances( | 51 | # return Component.get_instances( |
1388 | 52 | root=self.root, | 52 | # root=self.root, |
1389 | 53 | klass='IPython.core.iplib.InteractiveShell')[0] | 53 | # klass='IPython.core.iplib.InteractiveShell')[0] |
1390 | 54 | 54 | ||
1391 | 55 | def __enter__(self): | 55 | def __enter__(self): |
1392 | 56 | if self._nested_level == 0: | 56 | if self._nested_level == 0: |
1393 | 57 | 57 | ||
1394 | === modified file 'IPython/core/embed.py' | |||
1395 | --- IPython/core/embed.py 2009-09-13 22:06:04 +0000 | |||
1396 | +++ IPython/core/embed.py 2009-11-21 00:11:11 +0000 | |||
1397 | @@ -68,7 +68,7 @@ | |||
1398 | 68 | # is True by default. | 68 | # is True by default. |
1399 | 69 | display_banner = CBool(True) | 69 | display_banner = CBool(True) |
1400 | 70 | 70 | ||
1402 | 71 | def __init__(self, parent=None, config=None, ipythondir=None, usage=None, | 71 | def __init__(self, parent=None, config=None, ipython_dir=None, usage=None, |
1403 | 72 | user_ns=None, user_global_ns=None, | 72 | user_ns=None, user_global_ns=None, |
1404 | 73 | banner1=None, banner2=None, display_banner=None, | 73 | banner1=None, banner2=None, display_banner=None, |
1405 | 74 | custom_exceptions=((),None), exit_msg=''): | 74 | custom_exceptions=((),None), exit_msg=''): |
1406 | @@ -76,7 +76,7 @@ | |||
1407 | 76 | self.save_sys_ipcompleter() | 76 | self.save_sys_ipcompleter() |
1408 | 77 | 77 | ||
1409 | 78 | super(InteractiveShellEmbed,self).__init__( | 78 | super(InteractiveShellEmbed,self).__init__( |
1411 | 79 | parent=parent, config=config, ipythondir=ipythondir, usage=usage, | 79 | parent=parent, config=config, ipython_dir=ipython_dir, usage=usage, |
1412 | 80 | user_ns=user_ns, user_global_ns=user_global_ns, | 80 | user_ns=user_ns, user_global_ns=user_global_ns, |
1413 | 81 | banner1=banner1, banner2=banner2, display_banner=display_banner, | 81 | banner1=banner1, banner2=banner2, display_banner=display_banner, |
1414 | 82 | custom_exceptions=custom_exceptions) | 82 | custom_exceptions=custom_exceptions) |
1415 | @@ -233,14 +233,6 @@ | |||
1416 | 233 | for var in local_varnames: | 233 | for var in local_varnames: |
1417 | 234 | delvar(var,None) | 234 | delvar(var,None) |
1418 | 235 | 235 | ||
1419 | 236 | def set_completer_frame(self, frame=None): | ||
1420 | 237 | if frame: | ||
1421 | 238 | self.Completer.namespace = frame.f_locals | ||
1422 | 239 | self.Completer.global_namespace = frame.f_globals | ||
1423 | 240 | else: | ||
1424 | 241 | self.Completer.namespace = self.user_ns | ||
1425 | 242 | self.Completer.global_namespace = self.user_global_ns | ||
1426 | 243 | |||
1427 | 244 | 236 | ||
1428 | 245 | _embedded_shell = None | 237 | _embedded_shell = None |
1429 | 246 | 238 | ||
1430 | 247 | 239 | ||
1431 | === modified file 'IPython/core/ipapi.py' | |||
1432 | --- IPython/core/ipapi.py 2009-09-18 02:59:36 +0000 | |||
1433 | +++ IPython/core/ipapi.py 2009-11-21 00:11:11 +0000 | |||
1434 | @@ -18,16 +18,19 @@ | |||
1435 | 18 | # Imports | 18 | # Imports |
1436 | 19 | #----------------------------------------------------------------------------- | 19 | #----------------------------------------------------------------------------- |
1437 | 20 | 20 | ||
1439 | 21 | from IPython.core.error import TryNext, UsageError | 21 | from IPython.core.error import TryNext, UsageError, IPythonCoreError |
1440 | 22 | 22 | ||
1441 | 23 | #----------------------------------------------------------------------------- | 23 | #----------------------------------------------------------------------------- |
1442 | 24 | # Classes and functions | 24 | # Classes and functions |
1443 | 25 | #----------------------------------------------------------------------------- | 25 | #----------------------------------------------------------------------------- |
1444 | 26 | 26 | ||
1445 | 27 | |||
1446 | 27 | def get(): | 28 | def get(): |
1447 | 28 | """Get the most recently created InteractiveShell instance.""" | 29 | """Get the most recently created InteractiveShell instance.""" |
1448 | 29 | from IPython.core.iplib import InteractiveShell | 30 | from IPython.core.iplib import InteractiveShell |
1449 | 30 | insts = InteractiveShell.get_instances() | 31 | insts = InteractiveShell.get_instances() |
1450 | 32 | if len(insts)==0: | ||
1451 | 33 | return None | ||
1452 | 31 | most_recent = insts[0] | 34 | most_recent = insts[0] |
1453 | 32 | for inst in insts[1:]: | 35 | for inst in insts[1:]: |
1454 | 33 | if inst.created > most_recent.created: | 36 | if inst.created > most_recent.created: |
1455 | 34 | 37 | ||
1456 | === modified file 'IPython/core/ipapp.py' | |||
1457 | --- IPython/core/ipapp.py 2009-09-18 03:26:43 +0000 | |||
1458 | +++ IPython/core/ipapp.py 2009-11-21 00:11:11 +0000 | |||
1459 | @@ -1,7 +1,8 @@ | |||
1460 | 1 | #!/usr/bin/env python | 1 | #!/usr/bin/env python |
1461 | 2 | # encoding: utf-8 | 2 | # encoding: utf-8 |
1462 | 3 | """ | 3 | """ |
1464 | 4 | The main IPython application object | 4 | The :class:`~IPython.core.application.Application` object for the command |
1465 | 5 | line :command:`ipython` program. | ||
1466 | 5 | 6 | ||
1467 | 6 | Authors: | 7 | Authors: |
1468 | 7 | 8 | ||
1469 | @@ -28,19 +29,17 @@ | |||
1470 | 28 | import sys | 29 | import sys |
1471 | 29 | import warnings | 30 | import warnings |
1472 | 30 | 31 | ||
1474 | 31 | from IPython.core.application import Application, IPythonArgParseConfigLoader | 32 | from IPython.core.application import Application, BaseAppArgParseConfigLoader |
1475 | 32 | from IPython.core import release | 33 | from IPython.core import release |
1476 | 33 | from IPython.core.iplib import InteractiveShell | 34 | from IPython.core.iplib import InteractiveShell |
1477 | 34 | from IPython.config.loader import ( | 35 | from IPython.config.loader import ( |
1478 | 35 | NoConfigDefault, | 36 | NoConfigDefault, |
1479 | 36 | Config, | 37 | Config, |
1480 | 37 | ConfigError, | ||
1481 | 38 | PyFileConfigLoader | 38 | PyFileConfigLoader |
1482 | 39 | ) | 39 | ) |
1483 | 40 | 40 | ||
1484 | 41 | from IPython.lib import inputhook | 41 | from IPython.lib import inputhook |
1485 | 42 | 42 | ||
1486 | 43 | from IPython.utils.ipstruct import Struct | ||
1487 | 44 | from IPython.utils.genutils import filefind, get_ipython_dir | 43 | from IPython.utils.genutils import filefind, get_ipython_dir |
1488 | 45 | 44 | ||
1489 | 46 | #----------------------------------------------------------------------------- | 45 | #----------------------------------------------------------------------------- |
1490 | @@ -80,181 +79,181 @@ | |||
1491 | 80 | #----------------------------------------------------------------------------- | 79 | #----------------------------------------------------------------------------- |
1492 | 81 | 80 | ||
1493 | 82 | cl_args = ( | 81 | cl_args = ( |
1495 | 83 | (('-autocall',), dict( | 82 | (('--autocall',), dict( |
1496 | 84 | type=int, dest='InteractiveShell.autocall', default=NoConfigDefault, | 83 | type=int, dest='InteractiveShell.autocall', default=NoConfigDefault, |
1497 | 85 | help='Set the autocall value (0,1,2).', | 84 | help='Set the autocall value (0,1,2).', |
1498 | 86 | metavar='InteractiveShell.autocall') | 85 | metavar='InteractiveShell.autocall') |
1499 | 87 | ), | 86 | ), |
1501 | 88 | (('-autoindent',), dict( | 87 | (('--autoindent',), dict( |
1502 | 89 | action='store_true', dest='InteractiveShell.autoindent', default=NoConfigDefault, | 88 | action='store_true', dest='InteractiveShell.autoindent', default=NoConfigDefault, |
1503 | 90 | help='Turn on autoindenting.') | 89 | help='Turn on autoindenting.') |
1504 | 91 | ), | 90 | ), |
1506 | 92 | (('-noautoindent',), dict( | 91 | (('--no-autoindent',), dict( |
1507 | 93 | action='store_false', dest='InteractiveShell.autoindent', default=NoConfigDefault, | 92 | action='store_false', dest='InteractiveShell.autoindent', default=NoConfigDefault, |
1508 | 94 | help='Turn off autoindenting.') | 93 | help='Turn off autoindenting.') |
1509 | 95 | ), | 94 | ), |
1511 | 96 | (('-automagic',), dict( | 95 | (('--automagic',), dict( |
1512 | 97 | action='store_true', dest='InteractiveShell.automagic', default=NoConfigDefault, | 96 | action='store_true', dest='InteractiveShell.automagic', default=NoConfigDefault, |
1513 | 98 | help='Turn on the auto calling of magic commands.') | 97 | help='Turn on the auto calling of magic commands.') |
1514 | 99 | ), | 98 | ), |
1516 | 100 | (('-noautomagic',), dict( | 99 | (('--no-automagic',), dict( |
1517 | 101 | action='store_false', dest='InteractiveShell.automagic', default=NoConfigDefault, | 100 | action='store_false', dest='InteractiveShell.automagic', default=NoConfigDefault, |
1518 | 102 | help='Turn off the auto calling of magic commands.') | 101 | help='Turn off the auto calling of magic commands.') |
1519 | 103 | ), | 102 | ), |
1521 | 104 | (('-autoedit_syntax',), dict( | 103 | (('--autoedit-syntax',), dict( |
1522 | 105 | action='store_true', dest='InteractiveShell.autoedit_syntax', default=NoConfigDefault, | 104 | action='store_true', dest='InteractiveShell.autoedit_syntax', default=NoConfigDefault, |
1523 | 106 | help='Turn on auto editing of files with syntax errors.') | 105 | help='Turn on auto editing of files with syntax errors.') |
1524 | 107 | ), | 106 | ), |
1526 | 108 | (('-noautoedit_syntax',), dict( | 107 | (('--no-autoedit-syntax',), dict( |
1527 | 109 | action='store_false', dest='InteractiveShell.autoedit_syntax', default=NoConfigDefault, | 108 | action='store_false', dest='InteractiveShell.autoedit_syntax', default=NoConfigDefault, |
1528 | 110 | help='Turn off auto editing of files with syntax errors.') | 109 | help='Turn off auto editing of files with syntax errors.') |
1529 | 111 | ), | 110 | ), |
1531 | 112 | (('-banner',), dict( | 111 | (('--banner',), dict( |
1532 | 113 | action='store_true', dest='Global.display_banner', default=NoConfigDefault, | 112 | action='store_true', dest='Global.display_banner', default=NoConfigDefault, |
1533 | 114 | help='Display a banner upon starting IPython.') | 113 | help='Display a banner upon starting IPython.') |
1534 | 115 | ), | 114 | ), |
1536 | 116 | (('-nobanner',), dict( | 115 | (('--no-banner',), dict( |
1537 | 117 | action='store_false', dest='Global.display_banner', default=NoConfigDefault, | 116 | action='store_false', dest='Global.display_banner', default=NoConfigDefault, |
1538 | 118 | help="Don't display a banner upon starting IPython.") | 117 | help="Don't display a banner upon starting IPython.") |
1539 | 119 | ), | 118 | ), |
1541 | 120 | (('-cache_size',), dict( | 119 | (('--cache-size',), dict( |
1542 | 121 | type=int, dest='InteractiveShell.cache_size', default=NoConfigDefault, | 120 | type=int, dest='InteractiveShell.cache_size', default=NoConfigDefault, |
1543 | 122 | help="Set the size of the output cache.", | 121 | help="Set the size of the output cache.", |
1544 | 123 | metavar='InteractiveShell.cache_size') | 122 | metavar='InteractiveShell.cache_size') |
1545 | 124 | ), | 123 | ), |
1547 | 125 | (('-classic',), dict( | 124 | (('--classic',), dict( |
1548 | 126 | action='store_true', dest='Global.classic', default=NoConfigDefault, | 125 | action='store_true', dest='Global.classic', default=NoConfigDefault, |
1549 | 127 | help="Gives IPython a similar feel to the classic Python prompt.") | 126 | help="Gives IPython a similar feel to the classic Python prompt.") |
1550 | 128 | ), | 127 | ), |
1552 | 129 | (('-colors',), dict( | 128 | (('--colors',), dict( |
1553 | 130 | type=str, dest='InteractiveShell.colors', default=NoConfigDefault, | 129 | type=str, dest='InteractiveShell.colors', default=NoConfigDefault, |
1554 | 131 | help="Set the color scheme (NoColor, Linux, and LightBG).", | 130 | help="Set the color scheme (NoColor, Linux, and LightBG).", |
1555 | 132 | metavar='InteractiveShell.colors') | 131 | metavar='InteractiveShell.colors') |
1556 | 133 | ), | 132 | ), |
1558 | 134 | (('-color_info',), dict( | 133 | (('--color-info',), dict( |
1559 | 135 | action='store_true', dest='InteractiveShell.color_info', default=NoConfigDefault, | 134 | action='store_true', dest='InteractiveShell.color_info', default=NoConfigDefault, |
1560 | 136 | help="Enable using colors for info related things.") | 135 | help="Enable using colors for info related things.") |
1561 | 137 | ), | 136 | ), |
1563 | 138 | (('-nocolor_info',), dict( | 137 | (('--no-color-info',), dict( |
1564 | 139 | action='store_false', dest='InteractiveShell.color_info', default=NoConfigDefault, | 138 | action='store_false', dest='InteractiveShell.color_info', default=NoConfigDefault, |
1565 | 140 | help="Disable using colors for info related things.") | 139 | help="Disable using colors for info related things.") |
1566 | 141 | ), | 140 | ), |
1568 | 142 | (('-confirm_exit',), dict( | 141 | (('--confirm-exit',), dict( |
1569 | 143 | action='store_true', dest='InteractiveShell.confirm_exit', default=NoConfigDefault, | 142 | action='store_true', dest='InteractiveShell.confirm_exit', default=NoConfigDefault, |
1570 | 144 | help="Prompt the user when existing.") | 143 | help="Prompt the user when existing.") |
1571 | 145 | ), | 144 | ), |
1573 | 146 | (('-noconfirm_exit',), dict( | 145 | (('--no-confirm-exit',), dict( |
1574 | 147 | action='store_false', dest='InteractiveShell.confirm_exit', default=NoConfigDefault, | 146 | action='store_false', dest='InteractiveShell.confirm_exit', default=NoConfigDefault, |
1575 | 148 | help="Don't prompt the user when existing.") | 147 | help="Don't prompt the user when existing.") |
1576 | 149 | ), | 148 | ), |
1578 | 150 | (('-deep_reload',), dict( | 149 | (('--deep-reload',), dict( |
1579 | 151 | action='store_true', dest='InteractiveShell.deep_reload', default=NoConfigDefault, | 150 | action='store_true', dest='InteractiveShell.deep_reload', default=NoConfigDefault, |
1580 | 152 | help="Enable deep (recursive) reloading by default.") | 151 | help="Enable deep (recursive) reloading by default.") |
1581 | 153 | ), | 152 | ), |
1583 | 154 | (('-nodeep_reload',), dict( | 153 | (('--no-deep-reload',), dict( |
1584 | 155 | action='store_false', dest='InteractiveShell.deep_reload', default=NoConfigDefault, | 154 | action='store_false', dest='InteractiveShell.deep_reload', default=NoConfigDefault, |
1585 | 156 | help="Disable deep (recursive) reloading by default.") | 155 | help="Disable deep (recursive) reloading by default.") |
1586 | 157 | ), | 156 | ), |
1588 | 158 | (('-editor',), dict( | 157 | (('--editor',), dict( |
1589 | 159 | type=str, dest='InteractiveShell.editor', default=NoConfigDefault, | 158 | type=str, dest='InteractiveShell.editor', default=NoConfigDefault, |
1590 | 160 | help="Set the editor used by IPython (default to $EDITOR/vi/notepad).", | 159 | help="Set the editor used by IPython (default to $EDITOR/vi/notepad).", |
1591 | 161 | metavar='InteractiveShell.editor') | 160 | metavar='InteractiveShell.editor') |
1592 | 162 | ), | 161 | ), |
1594 | 163 | (('-log','-l'), dict( | 162 | (('--log','-l'), dict( |
1595 | 164 | action='store_true', dest='InteractiveShell.logstart', default=NoConfigDefault, | 163 | action='store_true', dest='InteractiveShell.logstart', default=NoConfigDefault, |
1596 | 165 | help="Start logging to the default file (./ipython_log.py).") | 164 | help="Start logging to the default file (./ipython_log.py).") |
1597 | 166 | ), | 165 | ), |
1600 | 167 | (('-logfile','-lf'), dict( | 166 | (('--logfile','-lf'), dict( |
1601 | 168 | type=str, dest='InteractiveShell.logfile', default=NoConfigDefault, | 167 | type=unicode, dest='InteractiveShell.logfile', default=NoConfigDefault, |
1602 | 169 | help="Start logging to logfile.", | 168 | help="Start logging to logfile.", |
1603 | 170 | metavar='InteractiveShell.logfile') | 169 | metavar='InteractiveShell.logfile') |
1604 | 171 | ), | 170 | ), |
1608 | 172 | (('-logappend','-la'), dict( | 171 | (('--log-append','-la'), dict( |
1609 | 173 | type=str, dest='InteractiveShell.logappend', default=NoConfigDefault, | 172 | type=unicode, dest='InteractiveShell.logappend', default=NoConfigDefault, |
1610 | 174 | help="Start logging to logappend in append mode.", | 173 | help="Start logging to the give file in append mode.", |
1611 | 175 | metavar='InteractiveShell.logfile') | 174 | metavar='InteractiveShell.logfile') |
1612 | 176 | ), | 175 | ), |
1614 | 177 | (('-pdb',), dict( | 176 | (('--pdb',), dict( |
1615 | 178 | action='store_true', dest='InteractiveShell.pdb', default=NoConfigDefault, | 177 | action='store_true', dest='InteractiveShell.pdb', default=NoConfigDefault, |
1616 | 179 | help="Enable auto calling the pdb debugger after every exception.") | 178 | help="Enable auto calling the pdb debugger after every exception.") |
1617 | 180 | ), | 179 | ), |
1619 | 181 | (('-nopdb',), dict( | 180 | (('--no-pdb',), dict( |
1620 | 182 | action='store_false', dest='InteractiveShell.pdb', default=NoConfigDefault, | 181 | action='store_false', dest='InteractiveShell.pdb', default=NoConfigDefault, |
1621 | 183 | help="Disable auto calling the pdb debugger after every exception.") | 182 | help="Disable auto calling the pdb debugger after every exception.") |
1622 | 184 | ), | 183 | ), |
1624 | 185 | (('-pprint',), dict( | 184 | (('--pprint',), dict( |
1625 | 186 | action='store_true', dest='InteractiveShell.pprint', default=NoConfigDefault, | 185 | action='store_true', dest='InteractiveShell.pprint', default=NoConfigDefault, |
1626 | 187 | help="Enable auto pretty printing of results.") | 186 | help="Enable auto pretty printing of results.") |
1627 | 188 | ), | 187 | ), |
1629 | 189 | (('-nopprint',), dict( | 188 | (('--no-pprint',), dict( |
1630 | 190 | action='store_false', dest='InteractiveShell.pprint', default=NoConfigDefault, | 189 | action='store_false', dest='InteractiveShell.pprint', default=NoConfigDefault, |
1631 | 191 | help="Disable auto auto pretty printing of results.") | 190 | help="Disable auto auto pretty printing of results.") |
1632 | 192 | ), | 191 | ), |
1634 | 193 | (('-prompt_in1','-pi1'), dict( | 192 | (('--prompt-in1','-pi1'), dict( |
1635 | 194 | type=str, dest='InteractiveShell.prompt_in1', default=NoConfigDefault, | 193 | type=str, dest='InteractiveShell.prompt_in1', default=NoConfigDefault, |
1636 | 195 | help="Set the main input prompt ('In [\#]: ')", | 194 | help="Set the main input prompt ('In [\#]: ')", |
1637 | 196 | metavar='InteractiveShell.prompt_in1') | 195 | metavar='InteractiveShell.prompt_in1') |
1638 | 197 | ), | 196 | ), |
1640 | 198 | (('-prompt_in2','-pi2'), dict( | 197 | (('--prompt-in2','-pi2'), dict( |
1641 | 199 | type=str, dest='InteractiveShell.prompt_in2', default=NoConfigDefault, | 198 | type=str, dest='InteractiveShell.prompt_in2', default=NoConfigDefault, |
1642 | 200 | help="Set the secondary input prompt (' .\D.: ')", | 199 | help="Set the secondary input prompt (' .\D.: ')", |
1643 | 201 | metavar='InteractiveShell.prompt_in2') | 200 | metavar='InteractiveShell.prompt_in2') |
1644 | 202 | ), | 201 | ), |
1646 | 203 | (('-prompt_out','-po'), dict( | 202 | (('--prompt-out','-po'), dict( |
1647 | 204 | type=str, dest='InteractiveShell.prompt_out', default=NoConfigDefault, | 203 | type=str, dest='InteractiveShell.prompt_out', default=NoConfigDefault, |
1648 | 205 | help="Set the output prompt ('Out[\#]:')", | 204 | help="Set the output prompt ('Out[\#]:')", |
1649 | 206 | metavar='InteractiveShell.prompt_out') | 205 | metavar='InteractiveShell.prompt_out') |
1650 | 207 | ), | 206 | ), |
1652 | 208 | (('-quick',), dict( | 207 | (('--quick',), dict( |
1653 | 209 | action='store_true', dest='Global.quick', default=NoConfigDefault, | 208 | action='store_true', dest='Global.quick', default=NoConfigDefault, |
1654 | 210 | help="Enable quick startup with no config files.") | 209 | help="Enable quick startup with no config files.") |
1655 | 211 | ), | 210 | ), |
1657 | 212 | (('-readline',), dict( | 211 | (('--readline',), dict( |
1658 | 213 | action='store_true', dest='InteractiveShell.readline_use', default=NoConfigDefault, | 212 | action='store_true', dest='InteractiveShell.readline_use', default=NoConfigDefault, |
1659 | 214 | help="Enable readline for command line usage.") | 213 | help="Enable readline for command line usage.") |
1660 | 215 | ), | 214 | ), |
1662 | 216 | (('-noreadline',), dict( | 215 | (('--no-readline',), dict( |
1663 | 217 | action='store_false', dest='InteractiveShell.readline_use', default=NoConfigDefault, | 216 | action='store_false', dest='InteractiveShell.readline_use', default=NoConfigDefault, |
1664 | 218 | help="Disable readline for command line usage.") | 217 | help="Disable readline for command line usage.") |
1665 | 219 | ), | 218 | ), |
1667 | 220 | (('-screen_length','-sl'), dict( | 219 | (('--screen-length','-sl'), dict( |
1668 | 221 | type=int, dest='InteractiveShell.screen_length', default=NoConfigDefault, | 220 | type=int, dest='InteractiveShell.screen_length', default=NoConfigDefault, |
1669 | 222 | help='Number of lines on screen, used to control printing of long strings.', | 221 | help='Number of lines on screen, used to control printing of long strings.', |
1670 | 223 | metavar='InteractiveShell.screen_length') | 222 | metavar='InteractiveShell.screen_length') |
1671 | 224 | ), | 223 | ), |
1673 | 225 | (('-separate_in','-si'), dict( | 224 | (('--separate-in','-si'), dict( |
1674 | 226 | type=str, dest='InteractiveShell.separate_in', default=NoConfigDefault, | 225 | type=str, dest='InteractiveShell.separate_in', default=NoConfigDefault, |
1675 | 227 | help="Separator before input prompts. Default '\n'.", | 226 | help="Separator before input prompts. Default '\n'.", |
1676 | 228 | metavar='InteractiveShell.separate_in') | 227 | metavar='InteractiveShell.separate_in') |
1677 | 229 | ), | 228 | ), |
1679 | 230 | (('-separate_out','-so'), dict( | 229 | (('--separate-out','-so'), dict( |
1680 | 231 | type=str, dest='InteractiveShell.separate_out', default=NoConfigDefault, | 230 | type=str, dest='InteractiveShell.separate_out', default=NoConfigDefault, |
1681 | 232 | help="Separator before output prompts. Default 0 (nothing).", | 231 | help="Separator before output prompts. Default 0 (nothing).", |
1682 | 233 | metavar='InteractiveShell.separate_out') | 232 | metavar='InteractiveShell.separate_out') |
1683 | 234 | ), | 233 | ), |
1685 | 235 | (('-separate_out2','-so2'), dict( | 234 | (('--separate-out2','-so2'), dict( |
1686 | 236 | type=str, dest='InteractiveShell.separate_out2', default=NoConfigDefault, | 235 | type=str, dest='InteractiveShell.separate_out2', default=NoConfigDefault, |
1687 | 237 | help="Separator after output prompts. Default 0 (nonight).", | 236 | help="Separator after output prompts. Default 0 (nonight).", |
1688 | 238 | metavar='InteractiveShell.separate_out2') | 237 | metavar='InteractiveShell.separate_out2') |
1689 | 239 | ), | 238 | ), |
1691 | 240 | (('-nosep',), dict( | 239 | (('-no-sep',), dict( |
1692 | 241 | action='store_true', dest='Global.nosep', default=NoConfigDefault, | 240 | action='store_true', dest='Global.nosep', default=NoConfigDefault, |
1693 | 242 | help="Eliminate all spacing between prompts.") | 241 | help="Eliminate all spacing between prompts.") |
1694 | 243 | ), | 242 | ), |
1696 | 244 | (('-term_title',), dict( | 243 | (('--term-title',), dict( |
1697 | 245 | action='store_true', dest='InteractiveShell.term_title', default=NoConfigDefault, | 244 | action='store_true', dest='InteractiveShell.term_title', default=NoConfigDefault, |
1698 | 246 | help="Enable auto setting the terminal title.") | 245 | help="Enable auto setting the terminal title.") |
1699 | 247 | ), | 246 | ), |
1701 | 248 | (('-noterm_title',), dict( | 247 | (('--no-term-title',), dict( |
1702 | 249 | action='store_false', dest='InteractiveShell.term_title', default=NoConfigDefault, | 248 | action='store_false', dest='InteractiveShell.term_title', default=NoConfigDefault, |
1703 | 250 | help="Disable auto setting the terminal title.") | 249 | help="Disable auto setting the terminal title.") |
1704 | 251 | ), | 250 | ), |
1706 | 252 | (('-xmode',), dict( | 251 | (('--xmode',), dict( |
1707 | 253 | type=str, dest='InteractiveShell.xmode', default=NoConfigDefault, | 252 | type=str, dest='InteractiveShell.xmode', default=NoConfigDefault, |
1708 | 254 | help="Exception mode ('Plain','Context','Verbose')", | 253 | help="Exception mode ('Plain','Context','Verbose')", |
1709 | 255 | metavar='InteractiveShell.xmode') | 254 | metavar='InteractiveShell.xmode') |
1710 | 256 | ), | 255 | ), |
1712 | 257 | (('-ext',), dict( | 256 | (('--ext',), dict( |
1713 | 258 | type=str, dest='Global.extra_extension', default=NoConfigDefault, | 257 | type=str, dest='Global.extra_extension', default=NoConfigDefault, |
1714 | 259 | help="The dotted module name of an IPython extension to load.", | 258 | help="The dotted module name of an IPython extension to load.", |
1715 | 260 | metavar='Global.extra_extension') | 259 | metavar='Global.extra_extension') |
1716 | @@ -268,36 +267,39 @@ | |||
1717 | 268 | action='store_true', dest='Global.force_interact', default=NoConfigDefault, | 267 | action='store_true', dest='Global.force_interact', default=NoConfigDefault, |
1718 | 269 | help="If running code from the command line, become interactive afterwards.") | 268 | help="If running code from the command line, become interactive afterwards.") |
1719 | 270 | ), | 269 | ), |
1721 | 271 | (('-wthread',), dict( | 270 | (('--wthread',), dict( |
1722 | 272 | action='store_true', dest='Global.wthread', default=NoConfigDefault, | 271 | action='store_true', dest='Global.wthread', default=NoConfigDefault, |
1723 | 273 | help="Enable wxPython event loop integration.") | 272 | help="Enable wxPython event loop integration.") |
1724 | 274 | ), | 273 | ), |
1726 | 275 | (('-q4thread','-qthread'), dict( | 274 | (('--q4thread','--qthread'), dict( |
1727 | 276 | action='store_true', dest='Global.q4thread', default=NoConfigDefault, | 275 | action='store_true', dest='Global.q4thread', default=NoConfigDefault, |
1728 | 277 | help="Enable Qt4 event loop integration. Qt3 is no longer supported.") | 276 | help="Enable Qt4 event loop integration. Qt3 is no longer supported.") |
1729 | 278 | ), | 277 | ), |
1731 | 279 | (('-gthread',), dict( | 278 | (('--gthread',), dict( |
1732 | 280 | action='store_true', dest='Global.gthread', default=NoConfigDefault, | 279 | action='store_true', dest='Global.gthread', default=NoConfigDefault, |
1733 | 281 | help="Enable GTK event loop integration.") | 280 | help="Enable GTK event loop integration.") |
1734 | 282 | ), | 281 | ), |
1735 | 283 | # # These are only here to get the proper deprecation warnings | 282 | # # These are only here to get the proper deprecation warnings |
1737 | 284 | (('-pylab',), dict( | 283 | (('--pylab',), dict( |
1738 | 285 | action='store_true', dest='Global.pylab', default=NoConfigDefault, | 284 | action='store_true', dest='Global.pylab', default=NoConfigDefault, |
1740 | 286 | help="Disabled. Pylab has been disabled until matplotlib supports this version of IPython.") | 285 | help="Disabled. Pylab has been disabled until matplotlib " |
1741 | 286 | "supports this version of IPython.") | ||
1742 | 287 | ) | 287 | ) |
1743 | 288 | ) | 288 | ) |
1744 | 289 | 289 | ||
1745 | 290 | 290 | ||
1747 | 291 | class IPythonAppCLConfigLoader(IPythonArgParseConfigLoader): | 291 | class IPythonAppCLConfigLoader(BaseAppArgParseConfigLoader): |
1748 | 292 | 292 | ||
1749 | 293 | arguments = cl_args | 293 | arguments = cl_args |
1750 | 294 | 294 | ||
1751 | 295 | 295 | ||
1753 | 296 | _default_config_file_name = 'ipython_config.py' | 296 | default_config_file_name = u'ipython_config.py' |
1754 | 297 | |||
1755 | 297 | 298 | ||
1756 | 298 | class IPythonApp(Application): | 299 | class IPythonApp(Application): |
1759 | 299 | name = 'ipython' | 300 | name = u'ipython' |
1760 | 300 | config_file_name = _default_config_file_name | 301 | description = 'IPython: an enhanced interactive Python shell.' |
1761 | 302 | config_file_name = default_config_file_name | ||
1762 | 301 | 303 | ||
1763 | 302 | def create_default_config(self): | 304 | def create_default_config(self): |
1764 | 303 | super(IPythonApp, self).create_default_config() | 305 | super(IPythonApp, self).create_default_config() |
1765 | @@ -313,11 +315,6 @@ | |||
1766 | 313 | # By default always interact by starting the IPython mainloop. | 315 | # By default always interact by starting the IPython mainloop. |
1767 | 314 | self.default_config.Global.interact = True | 316 | self.default_config.Global.interact = True |
1768 | 315 | 317 | ||
1769 | 316 | # Let the parent class set the default, but each time log_level | ||
1770 | 317 | # changes from config, we need to update self.log_level as that is | ||
1771 | 318 | # what updates the actual log level in self.log. | ||
1772 | 319 | self.default_config.Global.log_level = self.log_level | ||
1773 | 320 | |||
1774 | 321 | # No GUI integration by default | 318 | # No GUI integration by default |
1775 | 322 | self.default_config.Global.wthread = False | 319 | self.default_config.Global.wthread = False |
1776 | 323 | self.default_config.Global.q4thread = False | 320 | self.default_config.Global.q4thread = False |
1777 | @@ -326,8 +323,9 @@ | |||
1778 | 326 | def create_command_line_config(self): | 323 | def create_command_line_config(self): |
1779 | 327 | """Create and return a command line config loader.""" | 324 | """Create and return a command line config loader.""" |
1780 | 328 | return IPythonAppCLConfigLoader( | 325 | return IPythonAppCLConfigLoader( |
1783 | 329 | description=ipython_desc, | 326 | description=self.description, |
1784 | 330 | version=release.version) | 327 | version=release.version |
1785 | 328 | ) | ||
1786 | 331 | 329 | ||
1787 | 332 | def post_load_command_line_config(self): | 330 | def post_load_command_line_config(self): |
1788 | 333 | """Do actions after loading cl config.""" | 331 | """Do actions after loading cl config.""" |
1789 | @@ -477,9 +475,9 @@ | |||
1790 | 477 | self.shell.showtraceback() | 475 | self.shell.showtraceback() |
1791 | 478 | 476 | ||
1792 | 479 | def _exec_file(self, fname): | 477 | def _exec_file(self, fname): |
1794 | 480 | full_filename = filefind(fname, ['.', self.ipythondir]) | 478 | full_filename = filefind(fname, [u'.', self.ipython_dir]) |
1795 | 481 | if os.path.isfile(full_filename): | 479 | if os.path.isfile(full_filename): |
1797 | 482 | if full_filename.endswith('.py'): | 480 | if full_filename.endswith(u'.py'): |
1798 | 483 | self.log.info("Running file in user namespace: %s" % full_filename) | 481 | self.log.info("Running file in user namespace: %s" % full_filename) |
1799 | 484 | self.shell.safe_execfile(full_filename, self.shell.user_ns) | 482 | self.shell.safe_execfile(full_filename, self.shell.user_ns) |
1800 | 485 | elif full_filename.endswith('.ipy'): | 483 | elif full_filename.endswith('.ipy'): |
1801 | @@ -527,20 +525,20 @@ | |||
1802 | 527 | self.shell.mainloop() | 525 | self.shell.mainloop() |
1803 | 528 | 526 | ||
1804 | 529 | 527 | ||
1807 | 530 | def load_default_config(ipythondir=None): | 528 | def load_default_config(ipython_dir=None): |
1808 | 531 | """Load the default config file from the default ipythondir. | 529 | """Load the default config file from the default ipython_dir. |
1809 | 532 | 530 | ||
1810 | 533 | This is useful for embedded shells. | 531 | This is useful for embedded shells. |
1811 | 534 | """ | 532 | """ |
1815 | 535 | if ipythondir is None: | 533 | if ipython_dir is None: |
1816 | 536 | ipythondir = get_ipython_dir() | 534 | ipython_dir = get_ipython_dir() |
1817 | 537 | cl = PyFileConfigLoader(_default_config_file_name, ipythondir) | 535 | cl = PyFileConfigLoader(default_config_file_name, ipython_dir) |
1818 | 538 | config = cl.load_config() | 536 | config = cl.load_config() |
1819 | 539 | return config | 537 | return config |
1820 | 540 | 538 | ||
1821 | 541 | 539 | ||
1822 | 542 | def launch_new_instance(): | 540 | def launch_new_instance(): |
1824 | 543 | """Create a run a full blown IPython instance""" | 541 | """Create and run a full blown IPython instance""" |
1825 | 544 | app = IPythonApp() | 542 | app = IPythonApp() |
1826 | 545 | app.start() | 543 | app.start() |
1827 | 546 | 544 | ||
1828 | 547 | 545 | ||
1829 | === modified file 'IPython/core/iplib.py' | |||
1830 | --- IPython/core/iplib.py 2009-10-09 22:54:40 +0000 | |||
1831 | +++ IPython/core/iplib.py 2009-11-21 00:11:11 +0000 | |||
1832 | @@ -162,6 +162,15 @@ | |||
1833 | 162 | return ed | 162 | return ed |
1834 | 163 | 163 | ||
1835 | 164 | 164 | ||
1836 | 165 | def get_default_colors(): | ||
1837 | 166 | if sys.platform=='darwin': | ||
1838 | 167 | return "LightBG" | ||
1839 | 168 | elif os.name=='nt': | ||
1840 | 169 | return 'Linux' | ||
1841 | 170 | else: | ||
1842 | 171 | return 'Linux' | ||
1843 | 172 | |||
1844 | 173 | |||
1845 | 165 | class SeparateStr(Str): | 174 | class SeparateStr(Str): |
1846 | 166 | """A Str subclass to validate separate_in, separate_out, etc. | 175 | """A Str subclass to validate separate_in, separate_out, etc. |
1847 | 167 | 176 | ||
1848 | @@ -182,7 +191,7 @@ | |||
1849 | 182 | class InteractiveShell(Component, Magic): | 191 | class InteractiveShell(Component, Magic): |
1850 | 183 | """An enhanced, interactive shell for Python.""" | 192 | """An enhanced, interactive shell for Python.""" |
1851 | 184 | 193 | ||
1853 | 185 | autocall = Enum((0,1,2), config=True) | 194 | autocall = Enum((0,1,2), default_value=1, config=True) |
1854 | 186 | autoedit_syntax = CBool(False, config=True) | 195 | autoedit_syntax = CBool(False, config=True) |
1855 | 187 | autoindent = CBool(True, config=True) | 196 | autoindent = CBool(True, config=True) |
1856 | 188 | automagic = CBool(True, config=True) | 197 | automagic = CBool(True, config=True) |
1857 | @@ -192,7 +201,7 @@ | |||
1858 | 192 | cache_size = Int(1000, config=True) | 201 | cache_size = Int(1000, config=True) |
1859 | 193 | color_info = CBool(True, config=True) | 202 | color_info = CBool(True, config=True) |
1860 | 194 | colors = CaselessStrEnum(('NoColor','LightBG','Linux'), | 203 | colors = CaselessStrEnum(('NoColor','LightBG','Linux'), |
1862 | 195 | default_value='LightBG', config=True) | 204 | default_value=get_default_colors(), config=True) |
1863 | 196 | confirm_exit = CBool(True, config=True) | 205 | confirm_exit = CBool(True, config=True) |
1864 | 197 | debug = CBool(False, config=True) | 206 | debug = CBool(False, config=True) |
1865 | 198 | deep_reload = CBool(False, config=True) | 207 | deep_reload = CBool(False, config=True) |
1866 | @@ -206,7 +215,7 @@ | |||
1867 | 206 | embedded_active = CBool(False) | 215 | embedded_active = CBool(False) |
1868 | 207 | editor = Str(get_default_editor(), config=True) | 216 | editor = Str(get_default_editor(), config=True) |
1869 | 208 | filename = Str("<ipython console>") | 217 | filename = Str("<ipython console>") |
1871 | 209 | ipythondir= Unicode('', config=True) # Set to get_ipython_dir() in __init__ | 218 | ipython_dir= Unicode('', config=True) # Set to get_ipython_dir() in __init__ |
1872 | 210 | logstart = CBool(False, config=True) | 219 | logstart = CBool(False, config=True) |
1873 | 211 | logfile = Str('', config=True) | 220 | logfile = Str('', config=True) |
1874 | 212 | logappend = Str('', config=True) | 221 | logappend = Str('', config=True) |
1875 | @@ -264,7 +273,7 @@ | |||
1876 | 264 | # Subclasses with thread support should override this as needed. | 273 | # Subclasses with thread support should override this as needed. |
1877 | 265 | isthreaded = False | 274 | isthreaded = False |
1878 | 266 | 275 | ||
1880 | 267 | def __init__(self, parent=None, config=None, ipythondir=None, usage=None, | 276 | def __init__(self, parent=None, config=None, ipython_dir=None, usage=None, |
1881 | 268 | user_ns=None, user_global_ns=None, | 277 | user_ns=None, user_global_ns=None, |
1882 | 269 | banner1=None, banner2=None, display_banner=None, | 278 | banner1=None, banner2=None, display_banner=None, |
1883 | 270 | custom_exceptions=((),None)): | 279 | custom_exceptions=((),None)): |
1884 | @@ -274,7 +283,7 @@ | |||
1885 | 274 | super(InteractiveShell, self).__init__(parent, config=config) | 283 | super(InteractiveShell, self).__init__(parent, config=config) |
1886 | 275 | 284 | ||
1887 | 276 | # These are relatively independent and stateless | 285 | # These are relatively independent and stateless |
1889 | 277 | self.init_ipythondir(ipythondir) | 286 | self.init_ipython_dir(ipython_dir) |
1890 | 278 | self.init_instance_attrs() | 287 | self.init_instance_attrs() |
1891 | 279 | self.init_term_title() | 288 | self.init_term_title() |
1892 | 280 | self.init_usage(usage) | 289 | self.init_usage(usage) |
1893 | @@ -332,7 +341,7 @@ | |||
1894 | 332 | def _banner2_changed(self): | 341 | def _banner2_changed(self): |
1895 | 333 | self.compute_banner() | 342 | self.compute_banner() |
1896 | 334 | 343 | ||
1898 | 335 | def _ipythondir_changed(self, name, new): | 344 | def _ipython_dir_changed(self, name, new): |
1899 | 336 | if not os.path.isdir(new): | 345 | if not os.path.isdir(new): |
1900 | 337 | os.makedirs(new, mode = 0777) | 346 | os.makedirs(new, mode = 0777) |
1901 | 338 | if not os.path.isdir(self.ipython_extension_dir): | 347 | if not os.path.isdir(self.ipython_extension_dir): |
1902 | @@ -340,7 +349,7 @@ | |||
1903 | 340 | 349 | ||
1904 | 341 | @property | 350 | @property |
1905 | 342 | def ipython_extension_dir(self): | 351 | def ipython_extension_dir(self): |
1907 | 343 | return os.path.join(self.ipythondir, 'extensions') | 352 | return os.path.join(self.ipython_dir, 'extensions') |
1908 | 344 | 353 | ||
1909 | 345 | @property | 354 | @property |
1910 | 346 | def usable_screen_length(self): | 355 | def usable_screen_length(self): |
1911 | @@ -372,19 +381,19 @@ | |||
1912 | 372 | # init_* methods called by __init__ | 381 | # init_* methods called by __init__ |
1913 | 373 | #------------------------------------------------------------------------- | 382 | #------------------------------------------------------------------------- |
1914 | 374 | 383 | ||
1919 | 375 | def init_ipythondir(self, ipythondir): | 384 | def init_ipython_dir(self, ipython_dir): |
1920 | 376 | if ipythondir is not None: | 385 | if ipython_dir is not None: |
1921 | 377 | self.ipythondir = ipythondir | 386 | self.ipython_dir = ipython_dir |
1922 | 378 | self.config.Global.ipythondir = self.ipythondir | 387 | self.config.Global.ipython_dir = self.ipython_dir |
1923 | 379 | return | 388 | return |
1924 | 380 | 389 | ||
1927 | 381 | if hasattr(self.config.Global, 'ipythondir'): | 390 | if hasattr(self.config.Global, 'ipython_dir'): |
1928 | 382 | self.ipythondir = self.config.Global.ipythondir | 391 | self.ipython_dir = self.config.Global.ipython_dir |
1929 | 383 | else: | 392 | else: |
1931 | 384 | self.ipythondir = get_ipython_dir() | 393 | self.ipython_dir = get_ipython_dir() |
1932 | 385 | 394 | ||
1933 | 386 | # All children can just read this | 395 | # All children can just read this |
1935 | 387 | self.config.Global.ipythondir = self.ipythondir | 396 | self.config.Global.ipython_dir = self.ipython_dir |
1936 | 388 | 397 | ||
1937 | 389 | def init_instance_attrs(self): | 398 | def init_instance_attrs(self): |
1938 | 390 | self.jobs = BackgroundJobManager() | 399 | self.jobs = BackgroundJobManager() |
1939 | @@ -1070,7 +1079,7 @@ | |||
1940 | 1070 | histfname = 'history-%s' % self.profile | 1079 | histfname = 'history-%s' % self.profile |
1941 | 1071 | else: | 1080 | else: |
1942 | 1072 | histfname = 'history' | 1081 | histfname = 'history' |
1944 | 1073 | self.histfile = os.path.join(self.ipythondir, histfname) | 1082 | self.histfile = os.path.join(self.ipython_dir, histfname) |
1945 | 1074 | 1083 | ||
1946 | 1075 | # Fill the history zero entry, user counter starts at 1 | 1084 | # Fill the history zero entry, user counter starts at 1 |
1947 | 1076 | self.input_hist.append('\n') | 1085 | self.input_hist.append('\n') |
1948 | @@ -1078,12 +1087,12 @@ | |||
1949 | 1078 | 1087 | ||
1950 | 1079 | def init_shadow_hist(self): | 1088 | def init_shadow_hist(self): |
1951 | 1080 | try: | 1089 | try: |
1953 | 1081 | self.db = pickleshare.PickleShareDB(self.ipythondir + "/db") | 1090 | self.db = pickleshare.PickleShareDB(self.ipython_dir + "/db") |
1954 | 1082 | except exceptions.UnicodeDecodeError: | 1091 | except exceptions.UnicodeDecodeError: |
1956 | 1083 | print "Your ipythondir can't be decoded to unicode!" | 1092 | print "Your ipython_dir can't be decoded to unicode!" |
1957 | 1084 | print "Please set HOME environment variable to something that" | 1093 | print "Please set HOME environment variable to something that" |
1958 | 1085 | print r"only has ASCII characters, e.g. c:\home" | 1094 | print r"only has ASCII characters, e.g. c:\home" |
1960 | 1086 | print "Now it is", self.ipythondir | 1095 | print "Now it is", self.ipython_dir |
1961 | 1087 | sys.exit() | 1096 | sys.exit() |
1962 | 1088 | self.shadowhist = ipcorehist.ShadowHist(self.db) | 1097 | self.shadowhist = ipcorehist.ShadowHist(self.db) |
1963 | 1089 | 1098 | ||
1964 | @@ -1426,9 +1435,7 @@ | |||
1965 | 1426 | return outcomps | 1435 | return outcomps |
1966 | 1427 | 1436 | ||
1967 | 1428 | def set_custom_completer(self,completer,pos=0): | 1437 | def set_custom_completer(self,completer,pos=0): |
1971 | 1429 | """set_custom_completer(completer,pos=0) | 1438 | """Adds a new custom completer function. |
1969 | 1430 | |||
1970 | 1431 | Adds a new custom completer function. | ||
1972 | 1432 | 1439 | ||
1973 | 1433 | The position argument (defaults to 0) is the index in the completers | 1440 | The position argument (defaults to 0) is the index in the completers |
1974 | 1434 | list where you want the completer to be inserted.""" | 1441 | list where you want the completer to be inserted.""" |
1975 | @@ -1438,9 +1445,18 @@ | |||
1976 | 1438 | self.Completer.matchers.insert(pos,newcomp) | 1445 | self.Completer.matchers.insert(pos,newcomp) |
1977 | 1439 | 1446 | ||
1978 | 1440 | def set_completer(self): | 1447 | def set_completer(self): |
1980 | 1441 | """reset readline's completer to be our own.""" | 1448 | """Reset readline's completer to be our own.""" |
1981 | 1442 | self.readline.set_completer(self.Completer.complete) | 1449 | self.readline.set_completer(self.Completer.complete) |
1982 | 1443 | 1450 | ||
1983 | 1451 | def set_completer_frame(self, frame=None): | ||
1984 | 1452 | """Set the frame of the completer.""" | ||
1985 | 1453 | if frame: | ||
1986 | 1454 | self.Completer.namespace = frame.f_locals | ||
1987 | 1455 | self.Completer.global_namespace = frame.f_globals | ||
1988 | 1456 | else: | ||
1989 | 1457 | self.Completer.namespace = self.user_ns | ||
1990 | 1458 | self.Completer.global_namespace = self.user_global_ns | ||
1991 | 1459 | |||
1992 | 1444 | #------------------------------------------------------------------------- | 1460 | #------------------------------------------------------------------------- |
1993 | 1445 | # Things related to readline | 1461 | # Things related to readline |
1994 | 1446 | #------------------------------------------------------------------------- | 1462 | #------------------------------------------------------------------------- |
1995 | @@ -1913,7 +1929,7 @@ | |||
1996 | 1913 | # SystemExit exception changed between Python 2.4 and 2.5, so | 1929 | # SystemExit exception changed between Python 2.4 and 2.5, so |
1997 | 1914 | # the checks must be done in a version-dependent way. | 1930 | # the checks must be done in a version-dependent way. |
1998 | 1915 | show = False | 1931 | show = False |
2000 | 1916 | if status.message!=0 and not kw['exit_ignore']: | 1932 | if status.args[0]==0 and not kw['exit_ignore']: |
2001 | 1917 | show = True | 1933 | show = True |
2002 | 1918 | if show: | 1934 | if show: |
2003 | 1919 | self.showtraceback() | 1935 | self.showtraceback() |
2004 | @@ -2278,6 +2294,8 @@ | |||
2005 | 2278 | def get_component(self, name=None, klass=None): | 2294 | def get_component(self, name=None, klass=None): |
2006 | 2279 | """Fetch a component by name and klass in my tree.""" | 2295 | """Fetch a component by name and klass in my tree.""" |
2007 | 2280 | c = Component.get_instances(root=self, name=name, klass=klass) | 2296 | c = Component.get_instances(root=self, name=name, klass=klass) |
2008 | 2297 | if len(c) == 0: | ||
2009 | 2298 | return None | ||
2010 | 2281 | if len(c) == 1: | 2299 | if len(c) == 1: |
2011 | 2282 | return c[0] | 2300 | return c[0] |
2012 | 2283 | else: | 2301 | else: |
2013 | @@ -2309,7 +2327,7 @@ | |||
2014 | 2309 | You can put your extension modules anywhere you want, as long as | 2327 | You can put your extension modules anywhere you want, as long as |
2015 | 2310 | they can be imported by Python's standard import mechanism. However, | 2328 | they can be imported by Python's standard import mechanism. However, |
2016 | 2311 | to make it easy to write extensions, you can also put your extensions | 2329 | to make it easy to write extensions, you can also put your extensions |
2018 | 2312 | in ``os.path.join(self.ipythondir, 'extensions')``. This directory | 2330 | in ``os.path.join(self.ipython_dir, 'extensions')``. This directory |
2019 | 2313 | is added to ``sys.path`` automatically. | 2331 | is added to ``sys.path`` automatically. |
2020 | 2314 | """ | 2332 | """ |
2021 | 2315 | from IPython.utils.syspathcontext import prepended_to_syspath | 2333 | from IPython.utils.syspathcontext import prepended_to_syspath |
2022 | 2316 | 2334 | ||
2023 | === modified file 'IPython/core/magic.py' | |||
2024 | --- IPython/core/magic.py 2009-10-09 22:54:40 +0000 | |||
2025 | +++ IPython/core/magic.py 2009-11-21 00:11:11 +0000 | |||
2026 | @@ -21,6 +21,7 @@ | |||
2027 | 21 | import pdb | 21 | import pdb |
2028 | 22 | import pydoc | 22 | import pydoc |
2029 | 23 | import sys | 23 | import sys |
2030 | 24 | import shutil | ||
2031 | 24 | import re | 25 | import re |
2032 | 25 | import tempfile | 26 | import tempfile |
2033 | 26 | import time | 27 | import time |
2034 | @@ -1268,7 +1269,6 @@ | |||
2035 | 1268 | If you want IPython to automatically do this on every exception, see | 1269 | If you want IPython to automatically do this on every exception, see |
2036 | 1269 | the %pdb magic for more details. | 1270 | the %pdb magic for more details. |
2037 | 1270 | """ | 1271 | """ |
2038 | 1271 | |||
2039 | 1272 | self.shell.debugger(force=True) | 1272 | self.shell.debugger(force=True) |
2040 | 1273 | 1273 | ||
2041 | 1274 | @testdec.skip_doctest | 1274 | @testdec.skip_doctest |
2042 | @@ -3378,34 +3378,6 @@ | |||
2043 | 3378 | qr = IPython.core.usage.quick_reference + self.magic_magic('-brief') | 3378 | qr = IPython.core.usage.quick_reference + self.magic_magic('-brief') |
2044 | 3379 | 3379 | ||
2045 | 3380 | page(qr) | 3380 | page(qr) |
2046 | 3381 | |||
2047 | 3382 | def magic_upgrade(self,arg): | ||
2048 | 3383 | """ Upgrade your IPython installation | ||
2049 | 3384 | |||
2050 | 3385 | This will copy the config files that don't yet exist in your | ||
2051 | 3386 | ipython dir from the system config dir. Use this after upgrading | ||
2052 | 3387 | IPython if you don't wish to delete your .ipython dir. | ||
2053 | 3388 | |||
2054 | 3389 | Call with -nolegacy to get rid of ipythonrc* files (recommended for | ||
2055 | 3390 | new users) | ||
2056 | 3391 | |||
2057 | 3392 | """ | ||
2058 | 3393 | ip = self.getapi() | ||
2059 | 3394 | ipinstallation = path(IPython.__file__).dirname() | ||
2060 | 3395 | upgrade_script = '%s "%s"' % (sys.executable,ipinstallation / 'utils' / 'upgradedir.py') | ||
2061 | 3396 | src_config = ipinstallation / 'config' / 'userconfig' | ||
2062 | 3397 | userdir = path(ip.config.IPYTHONDIR) | ||
2063 | 3398 | cmd = '%s "%s" "%s"' % (upgrade_script, src_config, userdir) | ||
2064 | 3399 | print ">",cmd | ||
2065 | 3400 | shell(cmd) | ||
2066 | 3401 | if arg == '-nolegacy': | ||
2067 | 3402 | legacy = userdir.files('ipythonrc*') | ||
2068 | 3403 | print "Nuking legacy files:",legacy | ||
2069 | 3404 | |||
2070 | 3405 | [p.remove() for p in legacy] | ||
2071 | 3406 | suffix = (sys.platform == 'win32' and '.ini' or '') | ||
2072 | 3407 | (userdir / ('ipythonrc' + suffix)).write_text('# Empty, see ipy_user_conf.py\n') | ||
2073 | 3408 | |||
2074 | 3409 | 3381 | ||
2075 | 3410 | def magic_doctest_mode(self,parameter_s=''): | 3382 | def magic_doctest_mode(self,parameter_s=''): |
2076 | 3411 | """Toggle doctest mode on and off. | 3383 | """Toggle doctest mode on and off. |
2077 | @@ -3550,4 +3522,59 @@ | |||
2078 | 3550 | """Reload an IPython extension by its module name.""" | 3522 | """Reload an IPython extension by its module name.""" |
2079 | 3551 | self.reload_extension(module_str) | 3523 | self.reload_extension(module_str) |
2080 | 3552 | 3524 | ||
2081 | 3525 | def magic_install_profiles(self, s): | ||
2082 | 3526 | """Install the default IPython profiles into the .ipython dir. | ||
2083 | 3527 | |||
2084 | 3528 | If the default profiles have already been installed, they will not | ||
2085 | 3529 | be overwritten. You can force overwriting them by using the ``-o`` | ||
2086 | 3530 | option:: | ||
2087 | 3531 | |||
2088 | 3532 | In [1]: %install_profiles -o | ||
2089 | 3533 | """ | ||
2090 | 3534 | if '-o' in s: | ||
2091 | 3535 | overwrite = True | ||
2092 | 3536 | else: | ||
2093 | 3537 | overwrite = False | ||
2094 | 3538 | from IPython.config import profile | ||
2095 | 3539 | profile_dir = os.path.split(profile.__file__)[0] | ||
2096 | 3540 | ipython_dir = self.ipython_dir | ||
2097 | 3541 | files = os.listdir(profile_dir) | ||
2098 | 3542 | |||
2099 | 3543 | to_install = [] | ||
2100 | 3544 | for f in files: | ||
2101 | 3545 | if f.startswith('ipython_config'): | ||
2102 | 3546 | src = os.path.join(profile_dir, f) | ||
2103 | 3547 | dst = os.path.join(ipython_dir, f) | ||
2104 | 3548 | if (not os.path.isfile(dst)) or overwrite: | ||
2105 | 3549 | to_install.append((f, src, dst)) | ||
2106 | 3550 | if len(to_install)>0: | ||
2107 | 3551 | print "Installing profiles to: ", ipython_dir | ||
2108 | 3552 | for (f, src, dst) in to_install: | ||
2109 | 3553 | shutil.copy(src, dst) | ||
2110 | 3554 | print " %s" % f | ||
2111 | 3555 | |||
2112 | 3556 | def magic_install_default_config(self, s): | ||
2113 | 3557 | """Install IPython's default config file into the .ipython dir. | ||
2114 | 3558 | |||
2115 | 3559 | If the default config file (:file:`ipython_config.py`) is already | ||
2116 | 3560 | installed, it will not be overwritten. You can force overwriting | ||
2117 | 3561 | by using the ``-o`` option:: | ||
2118 | 3562 | |||
2119 | 3563 | In [1]: %install_default_config | ||
2120 | 3564 | """ | ||
2121 | 3565 | if '-o' in s: | ||
2122 | 3566 | overwrite = True | ||
2123 | 3567 | else: | ||
2124 | 3568 | overwrite = False | ||
2125 | 3569 | from IPython.config import default | ||
2126 | 3570 | config_dir = os.path.split(default.__file__)[0] | ||
2127 | 3571 | ipython_dir = self.ipython_dir | ||
2128 | 3572 | default_config_file_name = 'ipython_config.py' | ||
2129 | 3573 | src = os.path.join(config_dir, default_config_file_name) | ||
2130 | 3574 | dst = os.path.join(ipython_dir, default_config_file_name) | ||
2131 | 3575 | if (not os.path.isfile(dst)) or overwrite: | ||
2132 | 3576 | shutil.copy(src, dst) | ||
2133 | 3577 | print "Installing default config file: %s" % dst | ||
2134 | 3578 | |||
2135 | 3579 | |||
2136 | 3553 | # end Magic | 3580 | # end Magic |
2137 | 3554 | 3581 | ||
2138 | === removed file 'IPython/core/oldusersetup.py' | |||
2139 | --- IPython/core/oldusersetup.py 2009-08-18 06:55:22 +0000 | |||
2140 | +++ IPython/core/oldusersetup.py 1970-01-01 00:00:00 +0000 | |||
2141 | @@ -1,219 +0,0 @@ | |||
2142 | 1 | # -*- coding: utf-8 -*- | ||
2143 | 2 | """ | ||
2144 | 3 | Main IPython Component | ||
2145 | 4 | """ | ||
2146 | 5 | |||
2147 | 6 | #----------------------------------------------------------------------------- | ||
2148 | 7 | # Copyright (C) 2001 Janko Hauser <jhauser@zscout.de> | ||
2149 | 8 | # Copyright (C) 2001-2007 Fernando Perez. <fperez@colorado.edu> | ||
2150 | 9 | # Copyright (C) 2008-2009 The IPython Development Team | ||
2151 | 10 | # | ||
2152 | 11 | # Distributed under the terms of the BSD License. The full license is in | ||
2153 | 12 | # the file COPYING, distributed as part of this software. | ||
2154 | 13 | #----------------------------------------------------------------------------- | ||
2155 | 14 | |||
2156 | 15 | #----------------------------------------------------------------------------- | ||
2157 | 16 | # Imports | ||
2158 | 17 | #----------------------------------------------------------------------------- | ||
2159 | 18 | |||
2160 | 19 | import glob | ||
2161 | 20 | import os | ||
2162 | 21 | import shutil | ||
2163 | 22 | import sys | ||
2164 | 23 | |||
2165 | 24 | from IPython.utils.genutils import * | ||
2166 | 25 | |||
2167 | 26 | def user_setup(ipythondir,rc_suffix,mode='install',interactive=True): | ||
2168 | 27 | """Install or upgrade the user configuration directory. | ||
2169 | 28 | |||
2170 | 29 | Can be called when running for the first time or to upgrade the user's | ||
2171 | 30 | .ipython/ directory. | ||
2172 | 31 | |||
2173 | 32 | Parameters | ||
2174 | 33 | ---------- | ||
2175 | 34 | ipythondir : path | ||
2176 | 35 | The directory to be used for installation/upgrade. In 'install' mode, | ||
2177 | 36 | if this path already exists, the function exits immediately. | ||
2178 | 37 | |||
2179 | 38 | rc_suffix : str | ||
2180 | 39 | Extension for the config files. On *nix platforms it is typically the | ||
2181 | 40 | empty string, while Windows normally uses '.ini'. | ||
2182 | 41 | |||
2183 | 42 | mode : str, optional | ||
2184 | 43 | Valid modes are 'install' and 'upgrade'. | ||
2185 | 44 | |||
2186 | 45 | interactive : bool, optional | ||
2187 | 46 | If False, do not wait for user input on any errors. Normally after | ||
2188 | 47 | printing its status information, this function waits for the user to | ||
2189 | 48 | hit Return before proceeding. This is because the default use case is | ||
2190 | 49 | when first installing the IPython configuration, so we want the user to | ||
2191 | 50 | acknowledge the initial message, which contains some useful | ||
2192 | 51 | information. | ||
2193 | 52 | """ | ||
2194 | 53 | |||
2195 | 54 | # For automatic use, deactivate all i/o | ||
2196 | 55 | if interactive: | ||
2197 | 56 | def wait(): | ||
2198 | 57 | try: | ||
2199 | 58 | raw_input("Please press <RETURN> to start IPython.") | ||
2200 | 59 | except EOFError: | ||
2201 | 60 | print >> Term.cout | ||
2202 | 61 | print '*'*70 | ||
2203 | 62 | |||
2204 | 63 | def printf(s): | ||
2205 | 64 | print s | ||
2206 | 65 | else: | ||
2207 | 66 | wait = lambda : None | ||
2208 | 67 | printf = lambda s : None | ||
2209 | 68 | |||
2210 | 69 | # Install mode should be re-entrant: if the install dir already exists, | ||
2211 | 70 | # bail out cleanly. | ||
2212 | 71 | # XXX. This is too hasty to return. We need to check to make sure that | ||
2213 | 72 | # all the expected config files and directories are actually there. We | ||
2214 | 73 | # currently have a failure mode if someone deletes a needed config file | ||
2215 | 74 | # but still has the ipythondir. | ||
2216 | 75 | if mode == 'install' and os.path.isdir(ipythondir): | ||
2217 | 76 | return | ||
2218 | 77 | |||
2219 | 78 | cwd = os.getcwd() # remember where we started | ||
2220 | 79 | glb = glob.glob | ||
2221 | 80 | |||
2222 | 81 | printf('*'*70) | ||
2223 | 82 | if mode == 'install': | ||
2224 | 83 | printf( | ||
2225 | 84 | """Welcome to IPython. I will try to create a personal configuration directory | ||
2226 | 85 | where you can customize many aspects of IPython's functionality in:\n""") | ||
2227 | 86 | else: | ||
2228 | 87 | printf('I am going to upgrade your configuration in:') | ||
2229 | 88 | |||
2230 | 89 | printf(ipythondir) | ||
2231 | 90 | |||
2232 | 91 | rcdirend = os.path.join('IPython','config','userconfig') | ||
2233 | 92 | cfg = lambda d: os.path.join(d,rcdirend) | ||
2234 | 93 | try: | ||
2235 | 94 | rcdir = filter(os.path.isdir,map(cfg,sys.path))[0] | ||
2236 | 95 | printf("Initializing from configuration: %s" % rcdir) | ||
2237 | 96 | except IndexError: | ||
2238 | 97 | warning = """ | ||
2239 | 98 | Installation error. IPython's directory was not found. | ||
2240 | 99 | |||
2241 | 100 | Check the following: | ||
2242 | 101 | |||
2243 | 102 | The ipython/IPython directory should be in a directory belonging to your | ||
2244 | 103 | PYTHONPATH environment variable (that is, it should be in a directory | ||
2245 | 104 | belonging to sys.path). You can copy it explicitly there or just link to it. | ||
2246 | 105 | |||
2247 | 106 | IPython will create a minimal default configuration for you. | ||
2248 | 107 | |||
2249 | 108 | """ | ||
2250 | 109 | warn(warning) | ||
2251 | 110 | wait() | ||
2252 | 111 | |||
2253 | 112 | if sys.platform =='win32': | ||
2254 | 113 | inif = 'ipythonrc.ini' | ||
2255 | 114 | else: | ||
2256 | 115 | inif = 'ipythonrc' | ||
2257 | 116 | minimal_setup = {'ipy_user_conf.py' : 'import ipy_defaults', | ||
2258 | 117 | inif : '# intentionally left blank' } | ||
2259 | 118 | os.makedirs(ipythondir, mode = 0777) | ||
2260 | 119 | for f, cont in minimal_setup.items(): | ||
2261 | 120 | # In 2.5, this can be more cleanly done using 'with' | ||
2262 | 121 | fobj = file(ipythondir + '/' + f,'w') | ||
2263 | 122 | fobj.write(cont) | ||
2264 | 123 | fobj.close() | ||
2265 | 124 | |||
2266 | 125 | return | ||
2267 | 126 | |||
2268 | 127 | if mode == 'install': | ||
2269 | 128 | try: | ||
2270 | 129 | shutil.copytree(rcdir,ipythondir) | ||
2271 | 130 | os.chdir(ipythondir) | ||
2272 | 131 | rc_files = glb("ipythonrc*") | ||
2273 | 132 | for rc_file in rc_files: | ||
2274 | 133 | os.rename(rc_file,rc_file+rc_suffix) | ||
2275 | 134 | except: | ||
2276 | 135 | warning = """ | ||
2277 | 136 | |||
2278 | 137 | There was a problem with the installation: | ||
2279 | 138 | %s | ||
2280 | 139 | Try to correct it or contact the developers if you think it's a bug. | ||
2281 | 140 | IPython will proceed with builtin defaults.""" % sys.exc_info()[1] | ||
2282 | 141 | warn(warning) | ||
2283 | 142 | wait() | ||
2284 | 143 | return | ||
2285 | 144 | |||
2286 | 145 | elif mode == 'upgrade': | ||
2287 | 146 | try: | ||
2288 | 147 | os.chdir(ipythondir) | ||
2289 | 148 | except: | ||
2290 | 149 | printf(""" | ||
2291 | 150 | Can not upgrade: changing to directory %s failed. Details: | ||
2292 | 151 | %s | ||
2293 | 152 | """ % (ipythondir,sys.exc_info()[1]) ) | ||
2294 | 153 | wait() | ||
2295 | 154 | return | ||
2296 | 155 | else: | ||
2297 | 156 | sources = glb(os.path.join(rcdir,'[A-Za-z]*')) | ||
2298 | 157 | for new_full_path in sources: | ||
2299 | 158 | new_filename = os.path.basename(new_full_path) | ||
2300 | 159 | if new_filename.startswith('ipythonrc'): | ||
2301 | 160 | new_filename = new_filename + rc_suffix | ||
2302 | 161 | # The config directory should only contain files, skip any | ||
2303 | 162 | # directories which may be there (like CVS) | ||
2304 | 163 | if os.path.isdir(new_full_path): | ||
2305 | 164 | continue | ||
2306 | 165 | if os.path.exists(new_filename): | ||
2307 | 166 | old_file = new_filename+'.old' | ||
2308 | 167 | if os.path.exists(old_file): | ||
2309 | 168 | os.remove(old_file) | ||
2310 | 169 | os.rename(new_filename,old_file) | ||
2311 | 170 | shutil.copy(new_full_path,new_filename) | ||
2312 | 171 | else: | ||
2313 | 172 | raise ValueError('unrecognized mode for install: %r' % mode) | ||
2314 | 173 | |||
2315 | 174 | # Fix line-endings to those native to each platform in the config | ||
2316 | 175 | # directory. | ||
2317 | 176 | try: | ||
2318 | 177 | os.chdir(ipythondir) | ||
2319 | 178 | except: | ||
2320 | 179 | printf(""" | ||
2321 | 180 | Problem: changing to directory %s failed. | ||
2322 | 181 | Details: | ||
2323 | 182 | %s | ||
2324 | 183 | |||
2325 | 184 | Some configuration files may have incorrect line endings. This should not | ||
2326 | 185 | cause any problems during execution. """ % (ipythondir,sys.exc_info()[1]) ) | ||
2327 | 186 | wait() | ||
2328 | 187 | else: | ||
2329 | 188 | for fname in glb('ipythonrc*'): | ||
2330 | 189 | try: | ||
2331 | 190 | native_line_ends(fname,backup=0) | ||
2332 | 191 | except IOError: | ||
2333 | 192 | pass | ||
2334 | 193 | |||
2335 | 194 | if mode == 'install': | ||
2336 | 195 | printf(""" | ||
2337 | 196 | Successful installation! | ||
2338 | 197 | |||
2339 | 198 | Please read the sections 'Initial Configuration' and 'Quick Tips' in the | ||
2340 | 199 | IPython manual (there are both HTML and PDF versions supplied with the | ||
2341 | 200 | distribution) to make sure that your system environment is properly configured | ||
2342 | 201 | to take advantage of IPython's features. | ||
2343 | 202 | |||
2344 | 203 | Important note: the configuration system has changed! The old system is | ||
2345 | 204 | still in place, but its setting may be partly overridden by the settings in | ||
2346 | 205 | "~/.ipython/ipy_user_conf.py" config file. Please take a look at the file | ||
2347 | 206 | if some of the new settings bother you. | ||
2348 | 207 | |||
2349 | 208 | """) | ||
2350 | 209 | else: | ||
2351 | 210 | printf(""" | ||
2352 | 211 | Successful upgrade! | ||
2353 | 212 | |||
2354 | 213 | All files in your directory: | ||
2355 | 214 | %(ipythondir)s | ||
2356 | 215 | which would have been overwritten by the upgrade were backed up with a .old | ||
2357 | 216 | extension. If you had made particular customizations in those files you may | ||
2358 | 217 | want to merge them back into the new files.""" % locals() ) | ||
2359 | 218 | wait() | ||
2360 | 219 | os.chdir(cwd) | ||
2361 | 220 | \ No newline at end of file | 0 | \ No newline at end of file |
2362 | 221 | 1 | ||
2363 | === modified file 'IPython/core/prefilter.py' (properties changed: -x to +x) | |||
2364 | --- IPython/core/prefilter.py 2009-09-28 05:57:44 +0000 | |||
2365 | +++ IPython/core/prefilter.py 2009-11-21 00:11:11 +0000 | |||
2366 | @@ -39,7 +39,7 @@ | |||
2367 | 39 | from IPython.core.page import page | 39 | from IPython.core.page import page |
2368 | 40 | 40 | ||
2369 | 41 | from IPython.utils.traitlets import List, Int, Any, Str, CBool, Bool | 41 | from IPython.utils.traitlets import List, Int, Any, Str, CBool, Bool |
2371 | 42 | from IPython.utils.genutils import make_quoted_expr | 42 | from IPython.utils.genutils import make_quoted_expr, Term |
2372 | 43 | from IPython.utils.autoattr import auto_attr | 43 | from IPython.utils.autoattr import auto_attr |
2373 | 44 | 44 | ||
2374 | 45 | #----------------------------------------------------------------------------- | 45 | #----------------------------------------------------------------------------- |
2375 | 46 | 46 | ||
2376 | === modified file 'IPython/core/tests/test_iplib.py' | |||
2377 | --- IPython/core/tests/test_iplib.py 2009-08-19 21:56:41 +0000 | |||
2378 | +++ IPython/core/tests/test_iplib.py 2009-11-21 00:11:11 +0000 | |||
2379 | @@ -15,7 +15,7 @@ | |||
2380 | 15 | # our own packages | 15 | # our own packages |
2381 | 16 | from IPython.core import iplib | 16 | from IPython.core import iplib |
2382 | 17 | from IPython.core import ipapi | 17 | from IPython.core import ipapi |
2384 | 18 | from IPython.core.oldusersetup import user_setup | 18 | |
2385 | 19 | 19 | ||
2386 | 20 | #----------------------------------------------------------------------------- | 20 | #----------------------------------------------------------------------------- |
2387 | 21 | # Globals | 21 | # Globals |
2388 | @@ -54,27 +54,4 @@ | |||
2389 | 54 | continue | 54 | continue |
2390 | 55 | nt.assert_equals(len(ns),0) | 55 | nt.assert_equals(len(ns),0) |
2391 | 56 | 56 | ||
2392 | 57 | |||
2393 | 58 | # make sure that user_setup can be run re-entrantly in 'install' mode. | ||
2394 | 59 | def test_user_setup(): | ||
2395 | 60 | # use a lambda to pass kwargs to the generator | ||
2396 | 61 | user_setup = lambda a,k: user_setup(*a,**k) | ||
2397 | 62 | kw = dict(mode='install', interactive=False) | ||
2398 | 63 | |||
2399 | 64 | # Call the user setup and verify that the directory exists | ||
2400 | 65 | yield user_setup, (ip.config.IPYTHONDIR,''), kw | ||
2401 | 66 | yield os.path.isdir, ip.config.IPYTHONDIR | ||
2402 | 67 | |||
2403 | 68 | # Now repeat the operation with a non-existent directory. Check both that | ||
2404 | 69 | # the call succeeds and that the directory is created. | ||
2405 | 70 | tmpdir = tempfile.mktemp(prefix='ipython-test-') | ||
2406 | 71 | # Use a try with an empty except because try/finally doesn't work with a | ||
2407 | 72 | # yield in Python 2.4. | ||
2408 | 73 | try: | ||
2409 | 74 | yield user_setup, (tmpdir,''), kw | ||
2410 | 75 | yield os.path.isdir, tmpdir | ||
2411 | 76 | except: | ||
2412 | 77 | pass | ||
2413 | 78 | # Clean up the temp dir once done | ||
2414 | 79 | shutil.rmtree(tmpdir) | ||
2415 | 80 | 57 | ||
2416 | 81 | \ No newline at end of file | 58 | \ No newline at end of file |
2417 | 82 | 59 | ||
2418 | === modified file 'IPython/core/usage.py' | |||
2419 | --- IPython/core/usage.py 2009-08-18 23:09:56 +0000 | |||
2420 | +++ IPython/core/usage.py 2009-11-21 00:11:11 +0000 | |||
2421 | @@ -40,7 +40,7 @@ | |||
2422 | 40 | in directories. | 40 | in directories. |
2423 | 41 | 41 | ||
2424 | 42 | In the rest of this text, we will refer to this directory as | 42 | In the rest of this text, we will refer to this directory as |
2426 | 43 | IPYTHONDIR. | 43 | IPYTHON_DIR. |
2427 | 44 | 44 | ||
2428 | 45 | REGULAR OPTIONS | 45 | REGULAR OPTIONS |
2429 | 46 | After the above threading options have been given, regular options can | 46 | After the above threading options have been given, regular options can |
2430 | @@ -150,9 +150,9 @@ | |||
2431 | 150 | here (in case your default EDITOR is something like Emacs). | 150 | here (in case your default EDITOR is something like Emacs). |
2432 | 151 | 151 | ||
2433 | 152 | -ipythondir <name> | 152 | -ipythondir <name> |
2435 | 153 | The name of your IPython configuration directory IPYTHONDIR. | 153 | The name of your IPython configuration directory IPYTHON_DIR. |
2436 | 154 | This can also be specified through the environment variable | 154 | This can also be specified through the environment variable |
2438 | 155 | IPYTHONDIR. | 155 | IPYTHON_DIR. |
2439 | 156 | 156 | ||
2440 | 157 | -log|l Generate a log file of all input. The file is named | 157 | -log|l Generate a log file of all input. The file is named |
2441 | 158 | ipython_log.py in your current directory (which prevents logs | 158 | ipython_log.py in your current directory (which prevents logs |
2442 | @@ -201,10 +201,10 @@ | |||
2443 | 201 | 201 | ||
2444 | 202 | -profile|p <name> | 202 | -profile|p <name> |
2445 | 203 | Assume that your config file is ipythonrc-<name> (looks in cur- | 203 | Assume that your config file is ipythonrc-<name> (looks in cur- |
2447 | 204 | rent dir first, then in IPYTHONDIR). This is a quick way to keep | 204 | rent dir first, then in IPYTHON_DIR). This is a quick way to keep |
2448 | 205 | and load multiple config files for different tasks, especially | 205 | and load multiple config files for different tasks, especially |
2449 | 206 | if you use the include option of config files. You can keep a | 206 | if you use the include option of config files. You can keep a |
2451 | 207 | basic IPYTHONDIR/ipythonrc file and then have other 'profiles' | 207 | basic IPYTHON_DIR/ipythonrc file and then have other 'profiles' |
2452 | 208 | which include this one and load extra things for particular | 208 | which include this one and load extra things for particular |
2453 | 209 | tasks. For example: | 209 | tasks. For example: |
2454 | 210 | 210 | ||
2455 | @@ -245,7 +245,7 @@ | |||
2456 | 245 | -rcfile <name> | 245 | -rcfile <name> |
2457 | 246 | Name of your IPython resource configuration file. normally | 246 | Name of your IPython resource configuration file. normally |
2458 | 247 | IPython loads ipythonrc (from current directory) or | 247 | IPython loads ipythonrc (from current directory) or |
2460 | 248 | IPYTHONDIR/ipythonrc. If the loading of your config file fails, | 248 | IPYTHON_DIR/ipythonrc. If the loading of your config file fails, |
2461 | 249 | IPython starts with a bare bones configuration (no modules | 249 | IPython starts with a bare bones configuration (no modules |
2462 | 250 | loaded at all). | 250 | loaded at all). |
2463 | 251 | 251 | ||
2464 | @@ -284,7 +284,7 @@ | |||
2465 | 284 | Simply removes all input/output separators. | 284 | Simply removes all input/output separators. |
2466 | 285 | 285 | ||
2467 | 286 | -upgrade | 286 | -upgrade |
2469 | 287 | Allows you to upgrade your IPYTHONDIR configuration when you | 287 | Allows you to upgrade your IPYTHON_DIR configuration when you |
2470 | 288 | install a new version of IPython. Since new versions may | 288 | install a new version of IPython. Since new versions may |
2471 | 289 | include new command lines options or example files, this copies | 289 | include new command lines options or example files, this copies |
2472 | 290 | updated ipythonrc-type files. However, it backs up (with a .old | 290 | updated ipythonrc-type files. However, it backs up (with a .old |
2473 | 291 | 291 | ||
2474 | === renamed file 'IPython/kernel/magic.py' => 'IPython/extensions/parallelmagic.py' | |||
2475 | --- IPython/kernel/magic.py 2009-09-12 15:58:55 +0000 | |||
2476 | +++ IPython/extensions/parallelmagic.py 2009-11-21 00:11:11 +0000 | |||
2477 | @@ -1,171 +1,205 @@ | |||
2478 | 1 | #!/usr/bin/env python | ||
2479 | 1 | # encoding: utf-8 | 2 | # encoding: utf-8 |
2480 | 2 | 3 | ||
2481 | 3 | """Magic command interface for interactive parallel work.""" | 4 | """Magic command interface for interactive parallel work.""" |
2482 | 4 | 5 | ||
2487 | 5 | __docformat__ = "restructuredtext en" | 6 | #----------------------------------------------------------------------------- |
2488 | 6 | 7 | # Copyright (C) 2008-2009 The IPython Development Team | |
2485 | 7 | #------------------------------------------------------------------------------- | ||
2486 | 8 | # Copyright (C) 2008 The IPython Development Team | ||
2489 | 9 | # | 8 | # |
2490 | 10 | # Distributed under the terms of the BSD License. The full license is in | 9 | # Distributed under the terms of the BSD License. The full license is in |
2491 | 11 | # the file COPYING, distributed as part of this software. | 10 | # the file COPYING, distributed as part of this software. |
2493 | 12 | #------------------------------------------------------------------------------- | 11 | #----------------------------------------------------------------------------- |
2494 | 13 | 12 | ||
2496 | 14 | #------------------------------------------------------------------------------- | 13 | #----------------------------------------------------------------------------- |
2497 | 15 | # Imports | 14 | # Imports |
2499 | 16 | #------------------------------------------------------------------------------- | 15 | #----------------------------------------------------------------------------- |
2500 | 17 | 16 | ||
2501 | 18 | import new | 17 | import new |
2502 | 19 | 18 | ||
2510 | 20 | from IPython.core.iplib import InteractiveShell | 19 | from IPython.core.component import Component |
2511 | 21 | from IPython.core.shell import MTInteractiveShell | 20 | from IPython.utils.traitlets import Bool, Any |
2512 | 22 | 21 | from IPython.utils.autoattr import auto_attr | |
2513 | 23 | from twisted.internet.defer import Deferred | 22 | |
2514 | 24 | 23 | #----------------------------------------------------------------------------- | |
2508 | 25 | |||
2509 | 26 | #------------------------------------------------------------------------------- | ||
2515 | 27 | # Definitions of magic functions for use with IPython | 24 | # Definitions of magic functions for use with IPython |
2521 | 28 | #------------------------------------------------------------------------------- | 25 | #----------------------------------------------------------------------------- |
2522 | 29 | 26 | ||
2523 | 30 | NO_ACTIVE_CONTROLLER = """ | 27 | |
2524 | 31 | Error: No Controller is activated | 28 | NO_ACTIVE_MULTIENGINE_CLIENT = """ |
2525 | 32 | Use activate() on a RemoteController object to activate it for magics. | 29 | Use activate() on a MultiEngineClient object to activate it for magics. |
2526 | 33 | """ | 30 | """ |
2527 | 34 | 31 | ||
2552 | 35 | def magic_result(self,parameter_s=''): | 32 | |
2553 | 36 | """Print the result of command i on all engines of the active controller. | 33 | class ParalleMagicComponent(Component): |
2554 | 37 | 34 | """A component to manage the %result, %px and %autopx magics.""" | |
2555 | 38 | To activate a controller in IPython, first create it and then call | 35 | |
2556 | 39 | the activate() method. | 36 | active_multiengine_client = Any() |
2557 | 40 | 37 | verbose = Bool(False, config=True) | |
2558 | 41 | Then you can do the following: | 38 | |
2559 | 42 | 39 | def __init__(self, parent, name=None, config=None): | |
2560 | 43 | >>> result # Print the latest result | 40 | super(ParalleMagicComponent, self).__init__(parent, name=name, config=config) |
2561 | 44 | Printing result... | 41 | self._define_magics() |
2562 | 45 | [127.0.0.1:0] In [1]: b = 10 | 42 | # A flag showing if autopx is activated or not |
2563 | 46 | [127.0.0.1:1] In [1]: b = 10 | 43 | self.autopx = False |
2564 | 47 | 44 | ||
2565 | 48 | >>> result 0 # Print result 0 | 45 | # Access other components like this rather than by a regular attribute. |
2566 | 49 | In [14]: result 0 | 46 | # This won't lookup the InteractiveShell object until it is used and |
2567 | 50 | Printing result... | 47 | # then it is cached. This is both efficient and couples this class |
2568 | 51 | [127.0.0.1:0] In [0]: a = 5 | 48 | # more loosely to InteractiveShell. |
2569 | 52 | [127.0.0.1:1] In [0]: a = 5 | 49 | @auto_attr |
2570 | 53 | """ | 50 | def shell(self): |
2571 | 54 | try: | 51 | return Component.get_instances( |
2572 | 55 | activeController = __IPYTHON__.activeController | 52 | root=self.root, |
2573 | 56 | except AttributeError: | 53 | klass='IPython.core.iplib.InteractiveShell')[0] |
2574 | 57 | print NO_ACTIVE_CONTROLLER | 54 | |
2575 | 58 | else: | 55 | def _define_magics(self): |
2576 | 56 | """Define the magic functions.""" | ||
2577 | 57 | self.shell.define_magic('result', self.magic_result) | ||
2578 | 58 | self.shell.define_magic('px', self.magic_px) | ||
2579 | 59 | self.shell.define_magic('autopx', self.magic_autopx) | ||
2580 | 60 | |||
2581 | 61 | def magic_result(self, ipself, parameter_s=''): | ||
2582 | 62 | """Print the result of command i on all engines.. | ||
2583 | 63 | |||
2584 | 64 | To use this a :class:`MultiEngineClient` instance must be created | ||
2585 | 65 | and then activated by calling its :meth:`activate` method. | ||
2586 | 66 | |||
2587 | 67 | Then you can do the following:: | ||
2588 | 68 | |||
2589 | 69 | In [23]: %result | ||
2590 | 70 | Out[23]: | ||
2591 | 71 | <Results List> | ||
2592 | 72 | [0] In [6]: a = 10 | ||
2593 | 73 | [1] In [6]: a = 10 | ||
2594 | 74 | |||
2595 | 75 | In [22]: %result 6 | ||
2596 | 76 | Out[22]: | ||
2597 | 77 | <Results List> | ||
2598 | 78 | [0] In [6]: a = 10 | ||
2599 | 79 | [1] In [6]: a = 10 | ||
2600 | 80 | """ | ||
2601 | 81 | if self.active_multiengine_client is None: | ||
2602 | 82 | print NO_ACTIVE_MULTIENGINE_CLIENT | ||
2603 | 83 | return | ||
2604 | 84 | |||
2605 | 59 | try: | 85 | try: |
2606 | 60 | index = int(parameter_s) | 86 | index = int(parameter_s) |
2607 | 61 | except: | 87 | except: |
2608 | 62 | index = None | 88 | index = None |
2659 | 63 | result = activeController.get_result(index) | 89 | result = self.active_multiengine_client.get_result(index) |
2660 | 64 | return result | 90 | return result |
2661 | 65 | 91 | ||
2662 | 66 | def magic_px(self,parameter_s=''): | 92 | def magic_px(self, ipself, parameter_s=''): |
2663 | 67 | """Executes the given python command on the active IPython Controller. | 93 | """Executes the given python command in parallel. |
2664 | 68 | 94 | ||
2665 | 69 | To activate a Controller in IPython, first create it and then call | 95 | To use this a :class:`MultiEngineClient` instance must be created |
2666 | 70 | the activate() method. | 96 | and then activated by calling its :meth:`activate` method. |
2617 | 71 | |||
2618 | 72 | Then you can do the following: | ||
2619 | 73 | |||
2620 | 74 | >>> %px a = 5 # Runs a = 5 on all nodes | ||
2621 | 75 | """ | ||
2622 | 76 | |||
2623 | 77 | try: | ||
2624 | 78 | activeController = __IPYTHON__.activeController | ||
2625 | 79 | except AttributeError: | ||
2626 | 80 | print NO_ACTIVE_CONTROLLER | ||
2627 | 81 | else: | ||
2628 | 82 | print "Parallel execution on engines: %s" % activeController.targets | ||
2629 | 83 | result = activeController.execute(parameter_s) | ||
2630 | 84 | return result | ||
2631 | 85 | |||
2632 | 86 | def pxrunsource(self, source, filename="<input>", symbol="single"): | ||
2633 | 87 | |||
2634 | 88 | try: | ||
2635 | 89 | code = self.compile(source, filename, symbol) | ||
2636 | 90 | except (OverflowError, SyntaxError, ValueError): | ||
2637 | 91 | # Case 1 | ||
2638 | 92 | self.showsyntaxerror(filename) | ||
2639 | 93 | return None | ||
2640 | 94 | |||
2641 | 95 | if code is None: | ||
2642 | 96 | # Case 2 | ||
2643 | 97 | return True | ||
2644 | 98 | |||
2645 | 99 | # Case 3 | ||
2646 | 100 | # Because autopx is enabled, we now call executeAll or disable autopx if | ||
2647 | 101 | # %autopx or autopx has been called | ||
2648 | 102 | if 'get_ipython().magic("%autopx' in source or 'get_ipython().magic("autopx' in source: | ||
2649 | 103 | _disable_autopx(self) | ||
2650 | 104 | return False | ||
2651 | 105 | else: | ||
2652 | 106 | try: | ||
2653 | 107 | result = self.activeController.execute(source) | ||
2654 | 108 | except: | ||
2655 | 109 | self.showtraceback() | ||
2656 | 110 | else: | ||
2657 | 111 | print result.__repr__() | ||
2658 | 112 | return False | ||
2667 | 113 | 97 | ||
2687 | 114 | def magic_autopx(self, parameter_s=''): | 98 | Then you can do the following:: |
2688 | 115 | """Toggles auto parallel mode for the active IPython Controller. | 99 | |
2689 | 116 | 100 | In [24]: %px a = 5 | |
2690 | 117 | To activate a Controller in IPython, first create it and then call | 101 | Parallel execution on engines: all |
2691 | 118 | the activate() method. | 102 | Out[24]: |
2692 | 119 | 103 | <Results List> | |
2693 | 120 | Then you can do the following: | 104 | [0] In [7]: a = 5 |
2694 | 121 | 105 | [1] In [7]: a = 5 | |
2695 | 122 | >>> %autopx # Now all commands are executed in parallel | 106 | """ |
2696 | 123 | Auto Parallel Enabled | 107 | |
2697 | 124 | Type %autopx to disable | 108 | if self.active_multiengine_client is None: |
2698 | 125 | ... | 109 | print NO_ACTIVE_MULTIENGINE_CLIENT |
2699 | 126 | >>> %autopx # Now all commands are locally executed | 110 | return |
2700 | 127 | Auto Parallel Disabled | 111 | print "Parallel execution on engines: %s" % self.active_multiengine_client.targets |
2701 | 128 | """ | 112 | result = self.active_multiengine_client.execute(parameter_s) |
2702 | 129 | 113 | return result | |
2703 | 130 | if hasattr(self, 'autopx'): | 114 | |
2704 | 131 | if self.autopx == True: | 115 | def magic_autopx(self, ipself, parameter_s=''): |
2705 | 132 | _disable_autopx(self) | 116 | """Toggles auto parallel mode. |
2706 | 117 | |||
2707 | 118 | To use this a :class:`MultiEngineClient` instance must be created | ||
2708 | 119 | and then activated by calling its :meth:`activate` method. Once this | ||
2709 | 120 | is called, all commands typed at the command line are send to | ||
2710 | 121 | the engines to be executed in parallel. To control which engine | ||
2711 | 122 | are used, set the ``targets`` attributed of the multiengine client | ||
2712 | 123 | before entering ``%autopx`` mode. | ||
2713 | 124 | |||
2714 | 125 | Then you can do the following:: | ||
2715 | 126 | |||
2716 | 127 | In [25]: %autopx | ||
2717 | 128 | %autopx to enabled | ||
2718 | 129 | |||
2719 | 130 | In [26]: a = 10 | ||
2720 | 131 | <Results List> | ||
2721 | 132 | [0] In [8]: a = 10 | ||
2722 | 133 | [1] In [8]: a = 10 | ||
2723 | 134 | |||
2724 | 135 | |||
2725 | 136 | In [27]: %autopx | ||
2726 | 137 | %autopx disabled | ||
2727 | 138 | """ | ||
2728 | 139 | if self.autopx: | ||
2729 | 140 | self._disable_autopx() | ||
2730 | 133 | else: | 141 | else: |
2746 | 134 | _enable_autopx(self) | 142 | self._enable_autopx() |
2747 | 135 | else: | 143 | |
2748 | 136 | _enable_autopx(self) | 144 | def _enable_autopx(self): |
2749 | 137 | 145 | """Enable %autopx mode by saving the original runsource and installing | |
2750 | 138 | def _enable_autopx(self): | 146 | pxrunsource. |
2751 | 139 | """Enable %autopx mode by saving the original runsource and installing | 147 | """ |
2752 | 140 | pxrunsource. | 148 | if self.active_multiengine_client is None: |
2753 | 141 | """ | 149 | print NO_ACTIVE_MULTIENGINE_CLIENT |
2754 | 142 | try: | 150 | return |
2755 | 143 | activeController = __IPYTHON__.activeController | 151 | |
2756 | 144 | except AttributeError: | 152 | self._original_runsource = self.shell.runsource |
2757 | 145 | print "No active RemoteController found, use RemoteController.activate()." | 153 | self.shell.runsource = new.instancemethod( |
2758 | 146 | else: | 154 | self.pxrunsource, self.shell, self.shell.__class__ |
2759 | 147 | self._original_runsource = self.runsource | 155 | ) |
2745 | 148 | self.runsource = new.instancemethod(pxrunsource, self, self.__class__) | ||
2760 | 149 | self.autopx = True | 156 | self.autopx = True |
2768 | 150 | print "Auto Parallel Enabled\nType %autopx to disable" | 157 | print "%autopx enabled" |
2769 | 151 | 158 | ||
2770 | 152 | def _disable_autopx(self): | 159 | def _disable_autopx(self): |
2771 | 153 | """Disable %autopx by restoring the original runsource.""" | 160 | """Disable %autopx by restoring the original InteractiveShell.runsource.""" |
2772 | 154 | if hasattr(self, 'autopx'): | 161 | if self.autopx: |
2773 | 155 | if self.autopx == True: | 162 | self.shell.runsource = self._original_runsource |
2767 | 156 | self.runsource = self._original_runsource | ||
2774 | 157 | self.autopx = False | 163 | self.autopx = False |
2788 | 158 | print "Auto Parallel Disabled" | 164 | print "%autopx disabled" |
2789 | 159 | 165 | ||
2790 | 160 | # Add the new magic function to the class dict: | 166 | def pxrunsource(self, ipself, source, filename="<input>", symbol="single"): |
2791 | 161 | 167 | """A parallel replacement for InteractiveShell.runsource.""" | |
2792 | 162 | InteractiveShell.magic_result = magic_result | 168 | |
2793 | 163 | InteractiveShell.magic_px = magic_px | 169 | try: |
2794 | 164 | InteractiveShell.magic_autopx = magic_autopx | 170 | code = ipself.compile(source, filename, symbol) |
2795 | 165 | 171 | except (OverflowError, SyntaxError, ValueError): | |
2796 | 166 | # And remove the global name to keep global namespace clean. Don't worry, the | 172 | # Case 1 |
2797 | 167 | # copy bound to IPython stays, we're just removing the global name. | 173 | ipself.showsyntaxerror(filename) |
2798 | 168 | del magic_result | 174 | return None |
2799 | 169 | del magic_px | 175 | |
2800 | 170 | del magic_autopx | 176 | if code is None: |
2801 | 177 | # Case 2 | ||
2802 | 178 | return True | ||
2803 | 179 | |||
2804 | 180 | # Case 3 | ||
2805 | 181 | # Because autopx is enabled, we now call executeAll or disable autopx if | ||
2806 | 182 | # %autopx or autopx has been called | ||
2807 | 183 | if 'get_ipython().magic("%autopx' in source or 'get_ipython().magic("autopx' in source: | ||
2808 | 184 | self._disable_autopx() | ||
2809 | 185 | return False | ||
2810 | 186 | else: | ||
2811 | 187 | try: | ||
2812 | 188 | result = self.active_multiengine_client.execute(source) | ||
2813 | 189 | except: | ||
2814 | 190 | ipself.showtraceback() | ||
2815 | 191 | else: | ||
2816 | 192 | print result.__repr__() | ||
2817 | 193 | return False | ||
2818 | 194 | |||
2819 | 195 | |||
2820 | 196 | _loaded = False | ||
2821 | 197 | |||
2822 | 198 | |||
2823 | 199 | def load_ipython_extension(ip): | ||
2824 | 200 | """Load the extension in IPython.""" | ||
2825 | 201 | global _loaded | ||
2826 | 202 | if not _loaded: | ||
2827 | 203 | prd = ParalleMagicComponent(ip, name='parallel_magic') | ||
2828 | 204 | _loaded = True | ||
2829 | 171 | 205 | ||
2830 | 172 | 206 | ||
2831 | === modified file 'IPython/frontend/wx/ipythonx.py' | |||
2832 | --- IPython/frontend/wx/ipythonx.py 2009-03-29 01:05:30 +0000 | |||
2833 | +++ IPython/frontend/wx/ipythonx.py 2009-11-21 00:11:11 +0000 | |||
2834 | @@ -6,11 +6,10 @@ | |||
2835 | 6 | try: | 6 | try: |
2836 | 7 | import wx | 7 | import wx |
2837 | 8 | except ImportError, e: | 8 | except ImportError, e: |
2839 | 9 | e.message = """%s | 9 | e.args[0] = """%s |
2840 | 10 | ________________________________________________________________________________ | 10 | ________________________________________________________________________________ |
2841 | 11 | You need wxPython to run this application. | 11 | You need wxPython to run this application. |
2844 | 12 | """ % e.message | 12 | """ % e.args[0] |
2843 | 13 | e.args = (e.message, ) + e.args[1:] | ||
2845 | 14 | raise e | 13 | raise e |
2846 | 15 | 14 | ||
2847 | 16 | from wx_frontend import WxController | 15 | from wx_frontend import WxController |
2848 | 17 | 16 | ||
2849 | === modified file 'IPython/gui/wx/wxIPython.py' | |||
2850 | --- IPython/gui/wx/wxIPython.py 2009-08-19 21:56:41 +0000 | |||
2851 | +++ IPython/gui/wx/wxIPython.py 2009-11-21 00:11:11 +0000 | |||
2852 | @@ -109,7 +109,7 @@ | |||
2853 | 109 | 109 | ||
2854 | 110 | def optionSave(self, name, value): | 110 | def optionSave(self, name, value): |
2855 | 111 | ip = get() | 111 | ip = get() |
2857 | 112 | path = ip.config.IPYTHONDIR | 112 | path = ip.ipython_dir |
2858 | 113 | opt = open(path + '/options.conf','w') | 113 | opt = open(path + '/options.conf','w') |
2859 | 114 | 114 | ||
2860 | 115 | try: | 115 | try: |
2861 | @@ -126,7 +126,7 @@ | |||
2862 | 126 | def optionLoad(self): | 126 | def optionLoad(self): |
2863 | 127 | try: | 127 | try: |
2864 | 128 | ip = get() | 128 | ip = get() |
2866 | 129 | path = ip.config.IPYTHONDIR | 129 | path = ip.ipython_dir |
2867 | 130 | opt = open(path + '/options.conf','r') | 130 | opt = open(path + '/options.conf','r') |
2868 | 131 | lines = opt.readlines() | 131 | lines = opt.readlines() |
2869 | 132 | opt.close() | 132 | opt.close() |
2870 | 133 | 133 | ||
2871 | === modified file 'IPython/kernel/asyncclient.py' | |||
2872 | --- IPython/kernel/asyncclient.py 2008-07-22 02:39:52 +0000 | |||
2873 | +++ IPython/kernel/asyncclient.py 2009-11-21 00:11:11 +0000 | |||
2874 | @@ -1,3 +1,4 @@ | |||
2875 | 1 | #!/usr/bin/env python | ||
2876 | 1 | # encoding: utf-8 | 2 | # encoding: utf-8 |
2877 | 2 | 3 | ||
2878 | 3 | """Asynchronous clients for the IPython controller. | 4 | """Asynchronous clients for the IPython controller. |
2879 | @@ -9,32 +10,32 @@ | |||
2880 | 9 | 10 | ||
2881 | 10 | The main methods are are `get_*_client` and `get_client`. | 11 | The main methods are are `get_*_client` and `get_client`. |
2882 | 11 | """ | 12 | """ |
2888 | 12 | 13 | #----------------------------------------------------------------------------- | |
2889 | 13 | __docformat__ = "restructuredtext en" | 14 | # Copyright (C) 2008-2009 The IPython Development Team |
2885 | 14 | |||
2886 | 15 | #------------------------------------------------------------------------------- | ||
2887 | 16 | # Copyright (C) 2008 The IPython Development Team | ||
2890 | 17 | # | 15 | # |
2891 | 18 | # Distributed under the terms of the BSD License. The full license is in | 16 | # Distributed under the terms of the BSD License. The full license is in |
2892 | 19 | # the file COPYING, distributed as part of this software. | 17 | # the file COPYING, distributed as part of this software. |
2894 | 20 | #------------------------------------------------------------------------------- | 18 | #----------------------------------------------------------------------------- |
2895 | 21 | 19 | ||
2897 | 22 | #------------------------------------------------------------------------------- | 20 | #----------------------------------------------------------------------------- |
2898 | 23 | # Imports | 21 | # Imports |
2900 | 24 | #------------------------------------------------------------------------------- | 22 | #----------------------------------------------------------------------------- |
2901 | 25 | 23 | ||
2902 | 26 | from IPython.kernel import codeutil | 24 | from IPython.kernel import codeutil |
2904 | 27 | from IPython.kernel.clientconnector import ClientConnector | 25 | from IPython.kernel.clientconnector import ( |
2905 | 26 | AsyncClientConnector, | ||
2906 | 27 | AsyncCluster | ||
2907 | 28 | ) | ||
2908 | 28 | 29 | ||
2909 | 29 | # Other things that the user will need | 30 | # Other things that the user will need |
2910 | 30 | from IPython.kernel.task import MapTask, StringTask | 31 | from IPython.kernel.task import MapTask, StringTask |
2911 | 31 | from IPython.kernel.error import CompositeError | 32 | from IPython.kernel.error import CompositeError |
2912 | 32 | 33 | ||
2914 | 33 | #------------------------------------------------------------------------------- | 34 | #----------------------------------------------------------------------------- |
2915 | 34 | # Code | 35 | # Code |
2917 | 35 | #------------------------------------------------------------------------------- | 36 | #----------------------------------------------------------------------------- |
2918 | 36 | 37 | ||
2920 | 37 | _client_tub = ClientConnector() | 38 | _client_tub = AsyncClientConnector() |
2921 | 38 | get_multiengine_client = _client_tub.get_multiengine_client | 39 | get_multiengine_client = _client_tub.get_multiengine_client |
2922 | 39 | get_task_client = _client_tub.get_task_client | 40 | get_task_client = _client_tub.get_task_client |
2923 | 40 | get_client = _client_tub.get_client | 41 | get_client = _client_tub.get_client |
2924 | 41 | 42 | ||
2925 | === modified file 'IPython/kernel/client.py' | |||
2926 | --- IPython/kernel/client.py 2009-07-02 21:35:36 +0000 | |||
2927 | +++ IPython/kernel/client.py 2009-11-21 00:11:11 +0000 | |||
2928 | @@ -1,3 +1,4 @@ | |||
2929 | 1 | #!/usr/bin/env python | ||
2930 | 1 | # encoding: utf-8 | 2 | # encoding: utf-8 |
2931 | 2 | 3 | ||
2932 | 3 | """This module contains blocking clients for the controller interfaces. | 4 | """This module contains blocking clients for the controller interfaces. |
2933 | @@ -15,33 +16,36 @@ | |||
2934 | 15 | * CompositeError | 16 | * CompositeError |
2935 | 16 | """ | 17 | """ |
2936 | 17 | 18 | ||
2941 | 18 | __docformat__ = "restructuredtext en" | 19 | #----------------------------------------------------------------------------- |
2942 | 19 | 20 | # Copyright (C) 2008-2009 The IPython Development Team | |
2939 | 20 | #------------------------------------------------------------------------------- | ||
2940 | 21 | # Copyright (C) 2008 The IPython Development Team | ||
2943 | 22 | # | 21 | # |
2944 | 23 | # Distributed under the terms of the BSD License. The full license is in | 22 | # Distributed under the terms of the BSD License. The full license is in |
2945 | 24 | # the file COPYING, distributed as part of this software. | 23 | # the file COPYING, distributed as part of this software. |
2947 | 25 | #------------------------------------------------------------------------------- | 24 | #----------------------------------------------------------------------------- |
2948 | 26 | 25 | ||
2950 | 27 | #------------------------------------------------------------------------------- | 26 | #----------------------------------------------------------------------------- |
2951 | 28 | # Imports | 27 | # Imports |
2953 | 29 | #------------------------------------------------------------------------------- | 28 | #----------------------------------------------------------------------------- |
2954 | 30 | 29 | ||
2955 | 30 | from cStringIO import StringIO | ||
2956 | 31 | import sys | 31 | import sys |
2957 | 32 | import warnings | ||
2958 | 32 | 33 | ||
2959 | 33 | # from IPython.utils import growl | 34 | # from IPython.utils import growl |
2960 | 34 | # growl.start("IPython1 Client") | 35 | # growl.start("IPython1 Client") |
2961 | 35 | 36 | ||
2962 | 36 | 37 | ||
2963 | 37 | from twisted.internet import reactor | 38 | from twisted.internet import reactor |
2965 | 38 | from IPython.kernel.clientconnector import ClientConnector | 39 | from twisted.internet.error import PotentialZombieWarning |
2966 | 40 | from twisted.python import log | ||
2967 | 41 | |||
2968 | 42 | from IPython.kernel.clientconnector import ClientConnector, Cluster | ||
2969 | 39 | from IPython.kernel.twistedutil import ReactorInThread | 43 | from IPython.kernel.twistedutil import ReactorInThread |
2970 | 40 | from IPython.kernel.twistedutil import blockingCallFromThread | 44 | from IPython.kernel.twistedutil import blockingCallFromThread |
2971 | 41 | 45 | ||
2972 | 42 | # These enable various things | 46 | # These enable various things |
2973 | 43 | from IPython.kernel import codeutil | 47 | from IPython.kernel import codeutil |
2975 | 44 | import IPython.kernel.magic | 48 | # import IPython.kernel.magic |
2976 | 45 | 49 | ||
2977 | 46 | # Other things that the user will need | 50 | # Other things that the user will need |
2978 | 47 | from IPython.kernel.task import MapTask, StringTask | 51 | from IPython.kernel.task import MapTask, StringTask |
2979 | @@ -51,46 +55,34 @@ | |||
2980 | 51 | # Code | 55 | # Code |
2981 | 52 | #------------------------------------------------------------------------------- | 56 | #------------------------------------------------------------------------------- |
2982 | 53 | 57 | ||
2983 | 58 | warnings.simplefilter('ignore', PotentialZombieWarning) | ||
2984 | 59 | |||
2985 | 54 | _client_tub = ClientConnector() | 60 | _client_tub = ClientConnector() |
2986 | 55 | 61 | ||
3019 | 56 | 62 | get_multiengine_client = _client_tub.get_multiengine_client | |
3020 | 57 | def get_multiengine_client(furl_or_file=''): | 63 | get_task_client = _client_tub.get_task_client |
2989 | 58 | """Get the blocking MultiEngine client. | ||
2990 | 59 | |||
2991 | 60 | :Parameters: | ||
2992 | 61 | furl_or_file : str | ||
2993 | 62 | A furl or a filename containing a furl. If empty, the | ||
2994 | 63 | default furl_file will be used | ||
2995 | 64 | |||
2996 | 65 | :Returns: | ||
2997 | 66 | The connected MultiEngineClient instance | ||
2998 | 67 | """ | ||
2999 | 68 | client = blockingCallFromThread(_client_tub.get_multiengine_client, | ||
3000 | 69 | furl_or_file) | ||
3001 | 70 | return client.adapt_to_blocking_client() | ||
3002 | 71 | |||
3003 | 72 | def get_task_client(furl_or_file=''): | ||
3004 | 73 | """Get the blocking Task client. | ||
3005 | 74 | |||
3006 | 75 | :Parameters: | ||
3007 | 76 | furl_or_file : str | ||
3008 | 77 | A furl or a filename containing a furl. If empty, the | ||
3009 | 78 | default furl_file will be used | ||
3010 | 79 | |||
3011 | 80 | :Returns: | ||
3012 | 81 | The connected TaskClient instance | ||
3013 | 82 | """ | ||
3014 | 83 | client = blockingCallFromThread(_client_tub.get_task_client, | ||
3015 | 84 | furl_or_file) | ||
3016 | 85 | return client.adapt_to_blocking_client() | ||
3017 | 86 | |||
3018 | 87 | |||
3021 | 88 | MultiEngineClient = get_multiengine_client | 64 | MultiEngineClient = get_multiengine_client |
3022 | 89 | TaskClient = get_task_client | 65 | TaskClient = get_task_client |
3023 | 90 | 66 | ||
3025 | 91 | 67 | # This isn't great. I should probably set this up in the ReactorInThread | |
3026 | 68 | # class below. But, it does work for now. | ||
3027 | 69 | log.startLogging(sys.stdout, setStdout=0) | ||
3028 | 92 | 70 | ||
3029 | 93 | # Now we start the reactor in a thread | 71 | # Now we start the reactor in a thread |
3030 | 94 | rit = ReactorInThread() | 72 | rit = ReactorInThread() |
3031 | 95 | rit.setDaemon(True) | 73 | rit.setDaemon(True) |
3032 | 96 | rit.start() | ||
3033 | 97 | \ No newline at end of file | 74 | \ No newline at end of file |
3034 | 75 | rit.start() | ||
3035 | 76 | |||
3036 | 77 | |||
3037 | 78 | |||
3038 | 79 | |||
3039 | 80 | __all__ = [ | ||
3040 | 81 | 'MapTask', | ||
3041 | 82 | 'StringTask', | ||
3042 | 83 | 'MultiEngineClient', | ||
3043 | 84 | 'TaskClient', | ||
3044 | 85 | 'CompositeError', | ||
3045 | 86 | 'get_task_client', | ||
3046 | 87 | 'get_multiengine_client', | ||
3047 | 88 | 'Cluster' | ||
3048 | 89 | ] | ||
3049 | 98 | 90 | ||
3050 | === modified file 'IPython/kernel/clientconnector.py' | |||
3051 | --- IPython/kernel/clientconnector.py 2009-08-27 21:34:41 +0000 | |||
3052 | +++ IPython/kernel/clientconnector.py 2009-11-21 00:11:11 +0000 | |||
3053 | @@ -1,142 +1,268 @@ | |||
3054 | 1 | #!/usr/bin/env python | ||
3055 | 1 | # encoding: utf-8 | 2 | # encoding: utf-8 |
3056 | 2 | 3 | ||
3063 | 3 | """A class for handling client connections to the controller.""" | 4 | """Facilities for handling client connections to the controller.""" |
3064 | 4 | 5 | ||
3065 | 5 | __docformat__ = "restructuredtext en" | 6 | #----------------------------------------------------------------------------- |
3066 | 6 | 7 | # Copyright (C) 2008-2009 The IPython Development Team | |
3061 | 7 | #------------------------------------------------------------------------------- | ||
3062 | 8 | # Copyright (C) 2008 The IPython Development Team | ||
3067 | 9 | # | 8 | # |
3068 | 10 | # Distributed under the terms of the BSD License. The full license is in | 9 | # Distributed under the terms of the BSD License. The full license is in |
3069 | 11 | # the file COPYING, distributed as part of this software. | 10 | # the file COPYING, distributed as part of this software. |
3071 | 12 | #------------------------------------------------------------------------------- | 11 | #----------------------------------------------------------------------------- |
3072 | 13 | 12 | ||
3074 | 14 | #------------------------------------------------------------------------------- | 13 | #----------------------------------------------------------------------------- |
3075 | 15 | # Imports | 14 | # Imports |
3077 | 16 | #------------------------------------------------------------------------------- | 15 | #----------------------------------------------------------------------------- |
3078 | 16 | |||
3079 | 17 | from __future__ import with_statement | ||
3080 | 18 | import os | ||
3081 | 19 | |||
3082 | 20 | from IPython.kernel.fcutil import ( | ||
3083 | 21 | Tub, | ||
3084 | 22 | find_furl, | ||
3085 | 23 | is_valid_furl_or_file, | ||
3086 | 24 | validate_furl_or_file, | ||
3087 | 25 | FURLError | ||
3088 | 26 | ) | ||
3089 | 27 | from IPython.kernel.clusterdir import ClusterDir, ClusterDirError | ||
3090 | 28 | from IPython.kernel.launcher import IPClusterLauncher | ||
3091 | 29 | from IPython.kernel.twistedutil import ( | ||
3092 | 30 | gatherBoth, | ||
3093 | 31 | make_deferred, | ||
3094 | 32 | blockingCallFromThread, | ||
3095 | 33 | sleep_deferred | ||
3096 | 34 | ) | ||
3097 | 35 | from IPython.utils.importstring import import_item | ||
3098 | 36 | from IPython.utils.genutils import get_ipython_dir | ||
3099 | 17 | 37 | ||
3100 | 18 | from twisted.internet import defer | 38 | from twisted.internet import defer |
3112 | 19 | 39 | from twisted.internet.defer import inlineCallbacks, returnValue | |
3113 | 20 | from IPython.kernel.fcutil import Tub, UnauthenticatedTub | 40 | from twisted.python import failure, log |
3114 | 21 | 41 | ||
3115 | 22 | from IPython.kernel.config import config_manager as kernel_config_manager | 42 | #----------------------------------------------------------------------------- |
3105 | 23 | from IPython.utils.importstring import import_item | ||
3106 | 24 | from IPython.kernel.fcutil import find_furl | ||
3107 | 25 | |||
3108 | 26 | co = kernel_config_manager.get_config_obj() | ||
3109 | 27 | client_co = co['client'] | ||
3110 | 28 | |||
3111 | 29 | #------------------------------------------------------------------------------- | ||
3116 | 30 | # The ClientConnector class | 43 | # The ClientConnector class |
3127 | 31 | #------------------------------------------------------------------------------- | 44 | #----------------------------------------------------------------------------- |
3128 | 32 | 45 | ||
3129 | 33 | class ClientConnector(object): | 46 | DELAY = 0.2 |
3130 | 34 | """ | 47 | MAX_TRIES = 9 |
3131 | 35 | This class gets remote references from furls and returns the wrapped clients. | 48 | |
3132 | 36 | 49 | ||
3133 | 37 | This class is also used in `client.py` and `asyncclient.py` to create | 50 | class ClientConnectorError(Exception): |
3134 | 38 | a single per client-process Tub. | 51 | pass |
3135 | 39 | """ | 52 | |
3136 | 40 | 53 | ||
3137 | 54 | class AsyncClientConnector(object): | ||
3138 | 55 | """A class for getting remote references and clients from furls. | ||
3139 | 56 | |||
3140 | 57 | This start a single :class:`Tub` for all remote reference and caches | ||
3141 | 58 | references. | ||
3142 | 59 | """ | ||
3143 | 60 | |||
3144 | 41 | def __init__(self): | 61 | def __init__(self): |
3145 | 42 | self._remote_refs = {} | 62 | self._remote_refs = {} |
3146 | 43 | self.tub = Tub() | 63 | self.tub = Tub() |
3147 | 44 | self.tub.startService() | 64 | self.tub.startService() |
3149 | 45 | 65 | ||
3150 | 66 | def _find_furl(self, profile='default', cluster_dir=None, | ||
3151 | 67 | furl_or_file=None, furl_file_name=None, | ||
3152 | 68 | ipython_dir=None): | ||
3153 | 69 | """Find a FURL file by profile+ipython_dir or cluster dir. | ||
3154 | 70 | |||
3155 | 71 | This raises an :exc:`~IPython.kernel.fcutil.FURLError` exception | ||
3156 | 72 | if a FURL file can't be found. | ||
3157 | 73 | """ | ||
3158 | 74 | # Try by furl_or_file | ||
3159 | 75 | if furl_or_file is not None: | ||
3160 | 76 | validate_furl_or_file(furl_or_file) | ||
3161 | 77 | return furl_or_file | ||
3162 | 78 | |||
3163 | 79 | if furl_file_name is None: | ||
3164 | 80 | raise FURLError('A furl_file_name must be provided') | ||
3165 | 81 | |||
3166 | 82 | # Try by cluster_dir | ||
3167 | 83 | if cluster_dir is not None: | ||
3168 | 84 | cluster_dir_obj = ClusterDir.find_cluster_dir(cluster_dir) | ||
3169 | 85 | sdir = cluster_dir_obj.security_dir | ||
3170 | 86 | furl_file = os.path.join(sdir, furl_file_name) | ||
3171 | 87 | validate_furl_or_file(furl_file) | ||
3172 | 88 | return furl_file | ||
3173 | 89 | |||
3174 | 90 | # Try by profile | ||
3175 | 91 | if ipython_dir is None: | ||
3176 | 92 | ipython_dir = get_ipython_dir() | ||
3177 | 93 | if profile is not None: | ||
3178 | 94 | cluster_dir_obj = ClusterDir.find_cluster_dir_by_profile( | ||
3179 | 95 | ipython_dir, profile) | ||
3180 | 96 | sdir = cluster_dir_obj.security_dir | ||
3181 | 97 | furl_file = os.path.join(sdir, furl_file_name) | ||
3182 | 98 | validate_furl_or_file(furl_file) | ||
3183 | 99 | return furl_file | ||
3184 | 100 | |||
3185 | 101 | raise FURLError('Could not find a valid FURL file.') | ||
3186 | 102 | |||
3187 | 46 | def get_reference(self, furl_or_file): | 103 | def get_reference(self, furl_or_file): |
3191 | 47 | """ | 104 | """Get a remote reference using a furl or a file containing a furl. |
3192 | 48 | Get a remote reference using a furl or a file containing a furl. | 105 | |
3190 | 49 | |||
3193 | 50 | Remote references are cached locally so once a remote reference | 106 | Remote references are cached locally so once a remote reference |
3194 | 51 | has been retrieved for a given furl, the cached version is | 107 | has been retrieved for a given furl, the cached version is |
3195 | 52 | returned. | 108 | returned. |
3203 | 53 | 109 | ||
3204 | 54 | :Parameters: | 110 | Parameters |
3205 | 55 | furl_or_file : str | 111 | ---------- |
3206 | 56 | A furl or a filename containing a furl | 112 | furl_or_file : str |
3207 | 57 | 113 | A furl or a filename containing a furl. This should already be | |
3208 | 58 | :Returns: | 114 | validated, but might not yet exist. |
3209 | 59 | A deferred to a remote reference | 115 | |
3210 | 116 | Returns | ||
3211 | 117 | ------- | ||
3212 | 118 | A deferred to a remote reference | ||
3213 | 60 | """ | 119 | """ |
3215 | 61 | furl = find_furl(furl_or_file) | 120 | furl = furl_or_file |
3216 | 62 | if furl in self._remote_refs: | 121 | if furl in self._remote_refs: |
3217 | 63 | d = defer.succeed(self._remote_refs[furl]) | 122 | d = defer.succeed(self._remote_refs[furl]) |
3218 | 64 | else: | 123 | else: |
3219 | 65 | d = self.tub.getReference(furl) | 124 | d = self.tub.getReference(furl) |
3221 | 66 | d.addCallback(self.save_ref, furl) | 125 | d.addCallback(self._save_ref, furl) |
3222 | 67 | return d | 126 | return d |
3223 | 68 | 127 | ||
3228 | 69 | def save_ref(self, ref, furl): | 128 | def _save_ref(self, ref, furl): |
3229 | 70 | """ | 129 | """Cache a remote reference by its furl.""" |
3226 | 71 | Cache a remote reference by its furl. | ||
3227 | 72 | """ | ||
3230 | 73 | self._remote_refs[furl] = ref | 130 | self._remote_refs[furl] = ref |
3231 | 74 | return ref | 131 | return ref |
3232 | 75 | 132 | ||
3278 | 76 | def get_task_client(self, furl_or_file=''): | 133 | def get_task_client(self, profile='default', cluster_dir=None, |
3279 | 77 | """ | 134 | furl_or_file=None, ipython_dir=None, |
3280 | 78 | Get the task controller client. | 135 | delay=DELAY, max_tries=MAX_TRIES): |
3281 | 79 | 136 | """Get the task controller client. | |
3282 | 80 | This method is a simple wrapper around `get_client` that allow | 137 | |
3283 | 81 | `furl_or_file` to be empty, in which case, the furls is taken | 138 | This method is a simple wrapper around `get_client` that passes in |
3284 | 82 | from the default furl file given in the configuration. | 139 | the default name of the task client FURL file. Usually only |
3285 | 83 | 140 | the ``profile`` option will be needed. If a FURL file can't be | |
3286 | 84 | :Parameters: | 141 | found by its profile, use ``cluster_dir`` or ``furl_or_file``. |
3287 | 85 | furl_or_file : str | 142 | |
3288 | 86 | A furl or a filename containing a furl. If empty, the | 143 | Parameters |
3289 | 87 | default furl_file will be used | 144 | ---------- |
3290 | 88 | 145 | profile : str | |
3291 | 89 | :Returns: | 146 | The name of a cluster directory profile (default="default"). The |
3292 | 90 | A deferred to the actual client class | 147 | cluster directory "cluster_<profile>" will be searched for |
3293 | 91 | """ | 148 | in ``os.getcwd()``, the ipython_dir and then in the directories |
3294 | 92 | task_co = client_co['client_interfaces']['task'] | 149 | listed in the :env:`IPCLUSTER_DIR_PATH` environment variable. |
3295 | 93 | if furl_or_file: | 150 | cluster_dir : str |
3296 | 94 | ff = furl_or_file | 151 | The full path to a cluster directory. This is useful if profiles |
3297 | 95 | else: | 152 | are not being used. |
3298 | 96 | ff = task_co['furl_file'] | 153 | furl_or_file : str |
3299 | 97 | return self.get_client(ff) | 154 | A furl or a filename containing a FURLK. This is useful if you |
3300 | 98 | 155 | simply know the location of the FURL file. | |
3301 | 99 | def get_multiengine_client(self, furl_or_file=''): | 156 | ipython_dir : str |
3302 | 100 | """ | 157 | The location of the ipython_dir if different from the default. |
3303 | 101 | Get the multiengine controller client. | 158 | This is used if the cluster directory is being found by profile. |
3304 | 102 | 159 | delay : float | |
3305 | 103 | This method is a simple wrapper around `get_client` that allow | 160 | The initial delay between re-connection attempts. Susequent delays |
3306 | 104 | `furl_or_file` to be empty, in which case, the furls is taken | 161 | get longer according to ``delay[i] = 1.5*delay[i-1]``. |
3307 | 105 | from the default furl file given in the configuration. | 162 | max_tries : int |
3308 | 106 | 163 | The max number of re-connection attempts. | |
3309 | 107 | :Parameters: | 164 | |
3310 | 108 | furl_or_file : str | 165 | Returns |
3311 | 109 | A furl or a filename containing a furl. If empty, the | 166 | ------- |
3312 | 110 | default furl_file will be used | 167 | A deferred to the actual client class. |
3313 | 111 | 168 | """ | |
3314 | 112 | :Returns: | 169 | return self.get_client( |
3315 | 113 | A deferred to the actual client class | 170 | profile, cluster_dir, furl_or_file, |
3316 | 114 | """ | 171 | 'ipcontroller-tc.furl', ipython_dir, |
3317 | 115 | task_co = client_co['client_interfaces']['multiengine'] | 172 | delay, max_tries |
3318 | 116 | if furl_or_file: | 173 | ) |
3319 | 117 | ff = furl_or_file | 174 | |
3320 | 118 | else: | 175 | def get_multiengine_client(self, profile='default', cluster_dir=None, |
3321 | 119 | ff = task_co['furl_file'] | 176 | furl_or_file=None, ipython_dir=None, |
3322 | 120 | return self.get_client(ff) | 177 | delay=DELAY, max_tries=MAX_TRIES): |
3323 | 178 | """Get the multiengine controller client. | ||
3324 | 179 | |||
3325 | 180 | This method is a simple wrapper around `get_client` that passes in | ||
3326 | 181 | the default name of the task client FURL file. Usually only | ||
3327 | 182 | the ``profile`` option will be needed. If a FURL file can't be | ||
3328 | 183 | found by its profile, use ``cluster_dir`` or ``furl_or_file``. | ||
3329 | 184 | |||
3330 | 185 | Parameters | ||
3331 | 186 | ---------- | ||
3332 | 187 | profile : str | ||
3333 | 188 | The name of a cluster directory profile (default="default"). The | ||
3334 | 189 | cluster directory "cluster_<profile>" will be searched for | ||
3335 | 190 | in ``os.getcwd()``, the ipython_dir and then in the directories | ||
3336 | 191 | listed in the :env:`IPCLUSTER_DIR_PATH` environment variable. | ||
3337 | 192 | cluster_dir : str | ||
3338 | 193 | The full path to a cluster directory. This is useful if profiles | ||
3339 | 194 | are not being used. | ||
3340 | 195 | furl_or_file : str | ||
3341 | 196 | A furl or a filename containing a FURLK. This is useful if you | ||
3342 | 197 | simply know the location of the FURL file. | ||
3343 | 198 | ipython_dir : str | ||
3344 | 199 | The location of the ipython_dir if different from the default. | ||
3345 | 200 | This is used if the cluster directory is being found by profile. | ||
3346 | 201 | delay : float | ||
3347 | 202 | The initial delay between re-connection attempts. Susequent delays | ||
3348 | 203 | get longer according to ``delay[i] = 1.5*delay[i-1]``. | ||
3349 | 204 | max_tries : int | ||
3350 | 205 | The max number of re-connection attempts. | ||
3351 | 206 | |||
3352 | 207 | Returns | ||
3353 | 208 | ------- | ||
3354 | 209 | A deferred to the actual client class. | ||
3355 | 210 | """ | ||
3356 | 211 | return self.get_client( | ||
3357 | 212 | profile, cluster_dir, furl_or_file, | ||
3358 | 213 | 'ipcontroller-mec.furl', ipython_dir, | ||
3359 | 214 | delay, max_tries | ||
3360 | 215 | ) | ||
3361 | 121 | 216 | ||
3380 | 122 | def get_client(self, furl_or_file): | 217 | def get_client(self, profile='default', cluster_dir=None, |
3381 | 123 | """ | 218 | furl_or_file=None, furl_file_name=None, ipython_dir=None, |
3382 | 124 | Get a remote reference and wrap it in a client by furl. | 219 | delay=DELAY, max_tries=MAX_TRIES): |
3383 | 125 | 220 | """Get a remote reference and wrap it in a client by furl. | |
3384 | 126 | This method first gets a remote reference and then calls its | 221 | |
3385 | 127 | `get_client_name` method to find the apprpriate client class | 222 | This method is a simple wrapper around `get_client` that passes in |
3386 | 128 | that should be used to wrap the remote reference. | 223 | the default name of the task client FURL file. Usually only |
3387 | 129 | 224 | the ``profile`` option will be needed. If a FURL file can't be | |
3388 | 130 | :Parameters: | 225 | found by its profile, use ``cluster_dir`` or ``furl_or_file``. |
3389 | 131 | furl_or_file : str | 226 | |
3390 | 132 | A furl or a filename containing a furl | 227 | Parameters |
3391 | 133 | 228 | ---------- | |
3392 | 134 | :Returns: | 229 | profile : str |
3393 | 135 | A deferred to the actual client class | 230 | The name of a cluster directory profile (default="default"). The |
3394 | 136 | """ | 231 | cluster directory "cluster_<profile>" will be searched for |
3395 | 137 | furl = find_furl(furl_or_file) | 232 | in ``os.getcwd()``, the ipython_dir and then in the directories |
3396 | 138 | d = self.get_reference(furl) | 233 | listed in the :env:`IPCLUSTER_DIR_PATH` environment variable. |
3397 | 139 | def wrap_remote_reference(rr): | 234 | cluster_dir : str |
3398 | 235 | The full path to a cluster directory. This is useful if profiles | ||
3399 | 236 | are not being used. | ||
3400 | 237 | furl_or_file : str | ||
3401 | 238 | A furl or a filename containing a FURL. This is useful if you | ||
3402 | 239 | simply know the location of the FURL file. | ||
3403 | 240 | furl_file_name : str | ||
3404 | 241 | The filename (not the full path) of the FURL. This must be | ||
3405 | 242 | provided if ``furl_or_file`` is not. | ||
3406 | 243 | ipython_dir : str | ||
3407 | 244 | The location of the ipython_dir if different from the default. | ||
3408 | 245 | This is used if the cluster directory is being found by profile. | ||
3409 | 246 | delay : float | ||
3410 | 247 | The initial delay between re-connection attempts. Susequent delays | ||
3411 | 248 | get longer according to ``delay[i] = 1.5*delay[i-1]``. | ||
3412 | 249 | max_tries : int | ||
3413 | 250 | The max number of re-connection attempts. | ||
3414 | 251 | |||
3415 | 252 | Returns | ||
3416 | 253 | ------- | ||
3417 | 254 | A deferred to the actual client class. Or a failure to a | ||
3418 | 255 | :exc:`FURLError`. | ||
3419 | 256 | """ | ||
3420 | 257 | try: | ||
3421 | 258 | furl_file = self._find_furl( | ||
3422 | 259 | profile, cluster_dir, furl_or_file, | ||
3423 | 260 | furl_file_name, ipython_dir | ||
3424 | 261 | ) | ||
3425 | 262 | except FURLError: | ||
3426 | 263 | return defer.fail(failure.Failure()) | ||
3427 | 264 | |||
3428 | 265 | def _wrap_remote_reference(rr): | ||
3429 | 140 | d = rr.callRemote('get_client_name') | 266 | d = rr.callRemote('get_client_name') |
3430 | 141 | d.addCallback(lambda name: import_item(name)) | 267 | d.addCallback(lambda name: import_item(name)) |
3431 | 142 | def adapt(client_interface): | 268 | def adapt(client_interface): |
3432 | @@ -146,5 +272,502 @@ | |||
3433 | 146 | d.addCallback(adapt) | 272 | d.addCallback(adapt) |
3434 | 147 | 273 | ||
3435 | 148 | return d | 274 | return d |
3438 | 149 | d.addCallback(wrap_remote_reference) | 275 | |
3439 | 150 | return d | 276 | d = self._try_to_connect(furl_file, delay, max_tries, attempt=0) |
3440 | 277 | d.addCallback(_wrap_remote_reference) | ||
3441 | 278 | d.addErrback(self._handle_error, furl_file) | ||
3442 | 279 | return d | ||
3443 | 280 | |||
3444 | 281 | def _handle_error(self, f, furl_file): | ||
3445 | 282 | raise ClientConnectorError('Could not connect to the controller ' | ||
3446 | 283 | 'using the FURL file. This usually means that i) the controller ' | ||
3447 | 284 | 'was not started or ii) a firewall was blocking the client from ' | ||
3448 | 285 | 'connecting to the controller: %s' % furl_file) | ||
3449 | 286 | |||
3450 | 287 | @inlineCallbacks | ||
3451 | 288 | def _try_to_connect(self, furl_or_file, delay, max_tries, attempt): | ||
3452 | 289 | """Try to connect to the controller with retry logic.""" | ||
3453 | 290 | if attempt < max_tries: | ||
3454 | 291 | log.msg("Connecting [%r]" % attempt) | ||
3455 | 292 | try: | ||
3456 | 293 | self.furl = find_furl(furl_or_file) | ||
3457 | 294 | # Uncomment this to see the FURL being tried. | ||
3458 | 295 | # log.msg("FURL: %s" % self.furl) | ||
3459 | 296 | rr = yield self.get_reference(self.furl) | ||
3460 | 297 | log.msg("Connected: %s" % furl_or_file) | ||
3461 | 298 | except: | ||
3462 | 299 | if attempt==max_tries-1: | ||
3463 | 300 | # This will propagate the exception all the way to the top | ||
3464 | 301 | # where it can be handled. | ||
3465 | 302 | raise | ||
3466 | 303 | else: | ||
3467 | 304 | yield sleep_deferred(delay) | ||
3468 | 305 | rr = yield self._try_to_connect( | ||
3469 | 306 | furl_or_file, 1.5*delay, max_tries, attempt+1 | ||
3470 | 307 | ) | ||
3471 | 308 | returnValue(rr) | ||
3472 | 309 | else: | ||
3473 | 310 | returnValue(rr) | ||
3474 | 311 | else: | ||
3475 | 312 | raise ClientConnectorError( | ||
3476 | 313 | 'Could not connect to controller, max_tries (%r) exceeded. ' | ||
3477 | 314 | 'This usually means that i) the controller was not started, ' | ||
3478 | 315 | 'or ii) a firewall was blocking the client from connecting ' | ||
3479 | 316 | 'to the controller.' % max_tries | ||
3480 | 317 | ) | ||
3481 | 318 | |||
3482 | 319 | |||
3483 | 320 | class ClientConnector(object): | ||
3484 | 321 | """A blocking version of a client connector. | ||
3485 | 322 | |||
3486 | 323 | This class creates a single :class:`Tub` instance and allows remote | ||
3487 | 324 | references and client to be retrieved by their FURLs. Remote references | ||
3488 | 325 | are cached locally and FURL files can be found using profiles and cluster | ||
3489 | 326 | directories. | ||
3490 | 327 | """ | ||
3491 | 328 | |||
3492 | 329 | def __init__(self): | ||
3493 | 330 | self.async_cc = AsyncClientConnector() | ||
3494 | 331 | |||
3495 | 332 | def get_task_client(self, profile='default', cluster_dir=None, | ||
3496 | 333 | furl_or_file=None, ipython_dir=None, | ||
3497 | 334 | delay=DELAY, max_tries=MAX_TRIES): | ||
3498 | 335 | """Get the task client. | ||
3499 | 336 | |||
3500 | 337 | Usually only the ``profile`` option will be needed. If a FURL file | ||
3501 | 338 | can't be found by its profile, use ``cluster_dir`` or | ||
3502 | 339 | ``furl_or_file``. | ||
3503 | 340 | |||
3504 | 341 | Parameters | ||
3505 | 342 | ---------- | ||
3506 | 343 | profile : str | ||
3507 | 344 | The name of a cluster directory profile (default="default"). The | ||
3508 | 345 | cluster directory "cluster_<profile>" will be searched for | ||
3509 | 346 | in ``os.getcwd()``, the ipython_dir and then in the directories | ||
3510 | 347 | listed in the :env:`IPCLUSTER_DIR_PATH` environment variable. | ||
3511 | 348 | cluster_dir : str | ||
3512 | 349 | The full path to a cluster directory. This is useful if profiles | ||
3513 | 350 | are not being used. | ||
3514 | 351 | furl_or_file : str | ||
3515 | 352 | A furl or a filename containing a FURLK. This is useful if you | ||
3516 | 353 | simply know the location of the FURL file. | ||
3517 | 354 | ipython_dir : str | ||
3518 | 355 | The location of the ipython_dir if different from the default. | ||
3519 | 356 | This is used if the cluster directory is being found by profile. | ||
3520 | 357 | delay : float | ||
3521 | 358 | The initial delay between re-connection attempts. Susequent delays | ||
3522 | 359 | get longer according to ``delay[i] = 1.5*delay[i-1]``. | ||
3523 | 360 | max_tries : int | ||
3524 | 361 | The max number of re-connection attempts. | ||
3525 | 362 | |||
3526 | 363 | Returns | ||
3527 | 364 | ------- | ||
3528 | 365 | The task client instance. | ||
3529 | 366 | """ | ||
3530 | 367 | client = blockingCallFromThread( | ||
3531 | 368 | self.async_cc.get_task_client, profile, cluster_dir, | ||
3532 | 369 | furl_or_file, ipython_dir, delay, max_tries | ||
3533 | 370 | ) | ||
3534 | 371 | return client.adapt_to_blocking_client() | ||
3535 | 372 | |||
3536 | 373 | def get_multiengine_client(self, profile='default', cluster_dir=None, | ||
3537 | 374 | furl_or_file=None, ipython_dir=None, | ||
3538 | 375 | delay=DELAY, max_tries=MAX_TRIES): | ||
3539 | 376 | """Get the multiengine client. | ||
3540 | 377 | |||
3541 | 378 | Usually only the ``profile`` option will be needed. If a FURL file | ||
3542 | 379 | can't be found by its profile, use ``cluster_dir`` or | ||
3543 | 380 | ``furl_or_file``. | ||
3544 | 381 | |||
3545 | 382 | Parameters | ||
3546 | 383 | ---------- | ||
3547 | 384 | profile : str | ||
3548 | 385 | The name of a cluster directory profile (default="default"). The | ||
3549 | 386 | cluster directory "cluster_<profile>" will be searched for | ||
3550 | 387 | in ``os.getcwd()``, the ipython_dir and then in the directories | ||
3551 | 388 | listed in the :env:`IPCLUSTER_DIR_PATH` environment variable. | ||
3552 | 389 | cluster_dir : str | ||
3553 | 390 | The full path to a cluster directory. This is useful if profiles | ||
3554 | 391 | are not being used. | ||
3555 | 392 | furl_or_file : str | ||
3556 | 393 | A furl or a filename containing a FURLK. This is useful if you | ||
3557 | 394 | simply know the location of the FURL file. | ||
3558 | 395 | ipython_dir : str | ||
3559 | 396 | The location of the ipython_dir if different from the default. | ||
3560 | 397 | This is used if the cluster directory is being found by profile. | ||
3561 | 398 | delay : float | ||
3562 | 399 | The initial delay between re-connection attempts. Susequent delays | ||
3563 | 400 | get longer according to ``delay[i] = 1.5*delay[i-1]``. | ||
3564 | 401 | max_tries : int | ||
3565 | 402 | The max number of re-connection attempts. | ||
3566 | 403 | |||
3567 | 404 | Returns | ||
3568 | 405 | ------- | ||
3569 | 406 | The multiengine client instance. | ||
3570 | 407 | """ | ||
3571 | 408 | client = blockingCallFromThread( | ||
3572 | 409 | self.async_cc.get_multiengine_client, profile, cluster_dir, | ||
3573 | 410 | furl_or_file, ipython_dir, delay, max_tries | ||
3574 | 411 | ) | ||
3575 | 412 | return client.adapt_to_blocking_client() | ||
3576 | 413 | |||
3577 | 414 | def get_client(self, profile='default', cluster_dir=None, | ||
3578 | 415 | furl_or_file=None, ipython_dir=None, | ||
3579 | 416 | delay=DELAY, max_tries=MAX_TRIES): | ||
3580 | 417 | client = blockingCallFromThread( | ||
3581 | 418 | self.async_cc.get_client, profile, cluster_dir, | ||
3582 | 419 | furl_or_file, ipython_dir, | ||
3583 | 420 | delay, max_tries | ||
3584 | 421 | ) | ||
3585 | 422 | return client.adapt_to_blocking_client() | ||
3586 | 423 | |||
3587 | 424 | |||
3588 | 425 | class ClusterStateError(Exception): | ||
3589 | 426 | pass | ||
3590 | 427 | |||
3591 | 428 | |||
3592 | 429 | class AsyncCluster(object): | ||
3593 | 430 | """An class that wraps the :command:`ipcluster` script.""" | ||
3594 | 431 | |||
3595 | 432 | def __init__(self, profile='default', cluster_dir=None, ipython_dir=None, | ||
3596 | 433 | auto_create=False, auto_stop=True): | ||
3597 | 434 | """Create a class to manage an IPython cluster. | ||
3598 | 435 | |||
3599 | 436 | This class calls the :command:`ipcluster` command with the right | ||
3600 | 437 | options to start an IPython cluster. Typically a cluster directory | ||
3601 | 438 | must be created (:command:`ipcluster create`) and configured before | ||
3602 | 439 | using this class. Configuration is done by editing the | ||
3603 | 440 | configuration files in the top level of the cluster directory. | ||
3604 | 441 | |||
3605 | 442 | Parameters | ||
3606 | 443 | ---------- | ||
3607 | 444 | profile : str | ||
3608 | 445 | The name of a cluster directory profile (default="default"). The | ||
3609 | 446 | cluster directory "cluster_<profile>" will be searched for | ||
3610 | 447 | in ``os.getcwd()``, the ipython_dir and then in the directories | ||
3611 | 448 | listed in the :env:`IPCLUSTER_DIR_PATH` environment variable. | ||
3612 | 449 | cluster_dir : str | ||
3613 | 450 | The full path to a cluster directory. This is useful if profiles | ||
3614 | 451 | are not being used. | ||
3615 | 452 | ipython_dir : str | ||
3616 | 453 | The location of the ipython_dir if different from the default. | ||
3617 | 454 | This is used if the cluster directory is being found by profile. | ||
3618 | 455 | auto_create : bool | ||
3619 | 456 | Automatically create the cluster directory it is dones't exist. | ||
3620 | 457 | This will usually only make sense if using a local cluster | ||
3621 | 458 | (default=False). | ||
3622 | 459 | auto_stop : bool | ||
3623 | 460 | Automatically stop the cluster when this instance is garbage | ||
3624 | 461 | collected (default=True). This is useful if you want the cluster | ||
3625 | 462 | to live beyond your current process. There is also an instance | ||
3626 | 463 | attribute ``auto_stop`` to change this behavior. | ||
3627 | 464 | """ | ||
3628 | 465 | self._setup_cluster_dir(profile, cluster_dir, ipython_dir, auto_create) | ||
3629 | 466 | self.state = 'before' | ||
3630 | 467 | self.launcher = None | ||
3631 | 468 | self.client_connector = None | ||
3632 | 469 | self.auto_stop = auto_stop | ||
3633 | 470 | |||
3634 | 471 | def __del__(self): | ||
3635 | 472 | if self.auto_stop and self.state=='running': | ||
3636 | 473 | print "Auto stopping the cluster..." | ||
3637 | 474 | self.stop() | ||
3638 | 475 | |||
3639 | 476 | @property | ||
3640 | 477 | def location(self): | ||
3641 | 478 | if hasattr(self, 'cluster_dir_obj'): | ||
3642 | 479 | return self.cluster_dir_obj.location | ||
3643 | 480 | else: | ||
3644 | 481 | return '' | ||
3645 | 482 | |||
3646 | 483 | @property | ||
3647 | 484 | def running(self): | ||
3648 | 485 | if self.state=='running': | ||
3649 | 486 | return True | ||
3650 | 487 | else: | ||
3651 | 488 | return False | ||
3652 | 489 | |||
3653 | 490 | def _setup_cluster_dir(self, profile, cluster_dir, ipython_dir, auto_create): | ||
3654 | 491 | if ipython_dir is None: | ||
3655 | 492 | ipython_dir = get_ipython_dir() | ||
3656 | 493 | if cluster_dir is not None: | ||
3657 | 494 | try: | ||
3658 | 495 | self.cluster_dir_obj = ClusterDir.find_cluster_dir(cluster_dir) | ||
3659 | 496 | except ClusterDirError: | ||
3660 | 497 | pass | ||
3661 | 498 | if profile is not None: | ||
3662 | 499 | try: | ||
3663 | 500 | self.cluster_dir_obj = ClusterDir.find_cluster_dir_by_profile( | ||
3664 | 501 | ipython_dir, profile) | ||
3665 | 502 | except ClusterDirError: | ||
3666 | 503 | pass | ||
3667 | 504 | if auto_create or profile=='default': | ||
3668 | 505 | # This should call 'ipcluster create --profile default | ||
3669 | 506 | self.cluster_dir_obj = ClusterDir.create_cluster_dir_by_profile( | ||
3670 | 507 | ipython_dir, profile) | ||
3671 | 508 | else: | ||
3672 | 509 | raise ClusterDirError('Cluster dir not found.') | ||
3673 | 510 | |||
3674 | 511 | @make_deferred | ||
3675 | 512 | def start(self, n=2): | ||
3676 | 513 | """Start the IPython cluster with n engines. | ||
3677 | 514 | |||
3678 | 515 | Parameters | ||
3679 | 516 | ---------- | ||
3680 | 517 | n : int | ||
3681 | 518 | The number of engine to start. | ||
3682 | 519 | """ | ||
3683 | 520 | # We might want to add logic to test if the cluster has started | ||
3684 | 521 | # by another process.... | ||
3685 | 522 | if not self.state=='running': | ||
3686 | 523 | self.launcher = IPClusterLauncher(os.getcwd()) | ||
3687 | 524 | self.launcher.ipcluster_n = n | ||
3688 | 525 | self.launcher.ipcluster_subcommand = 'start' | ||
3689 | 526 | d = self.launcher.start() | ||
3690 | 527 | d.addCallback(self._handle_start) | ||
3691 | 528 | return d | ||
3692 | 529 | else: | ||
3693 | 530 | raise ClusterStateError('Cluster is already running') | ||
3694 | 531 | |||
3695 | 532 | @make_deferred | ||
3696 | 533 | def stop(self): | ||
3697 | 534 | """Stop the IPython cluster if it is running.""" | ||
3698 | 535 | if self.state=='running': | ||
3699 | 536 | d1 = self.launcher.observe_stop() | ||
3700 | 537 | d1.addCallback(self._handle_stop) | ||
3701 | 538 | d2 = self.launcher.stop() | ||
3702 | 539 | return gatherBoth([d1, d2], consumeErrors=True) | ||
3703 | 540 | else: | ||
3704 | 541 | raise ClusterStateError("Cluster not running") | ||
3705 | 542 | |||
3706 | 543 | def get_multiengine_client(self, delay=DELAY, max_tries=MAX_TRIES): | ||
3707 | 544 | """Get the multiengine client for the running cluster. | ||
3708 | 545 | |||
3709 | 546 | If this fails, it means that the cluster has not finished starting. | ||
3710 | 547 | Usually waiting a few seconds are re-trying will solve this. | ||
3711 | 548 | """ | ||
3712 | 549 | if self.client_connector is None: | ||
3713 | 550 | self.client_connector = AsyncClientConnector() | ||
3714 | 551 | return self.client_connector.get_multiengine_client( | ||
3715 | 552 | cluster_dir=self.cluster_dir_obj.location, | ||
3716 | 553 | delay=delay, max_tries=max_tries | ||
3717 | 554 | ) | ||
3718 | 555 | |||
3719 | 556 | def get_task_client(self, delay=DELAY, max_tries=MAX_TRIES): | ||
3720 | 557 | """Get the task client for the running cluster. | ||
3721 | 558 | |||
3722 | 559 | If this fails, it means that the cluster has not finished starting. | ||
3723 | 560 | Usually waiting a few seconds are re-trying will solve this. | ||
3724 | 561 | """ | ||
3725 | 562 | if self.client_connector is None: | ||
3726 | 563 | self.client_connector = AsyncClientConnector() | ||
3727 | 564 | return self.client_connector.get_task_client( | ||
3728 | 565 | cluster_dir=self.cluster_dir_obj.location, | ||
3729 | 566 | delay=delay, max_tries=max_tries | ||
3730 | 567 | ) | ||
3731 | 568 | |||
3732 | 569 | def get_ipengine_logs(self): | ||
3733 | 570 | return self.get_logs_by_name('ipengine') | ||
3734 | 571 | |||
3735 | 572 | def get_ipcontroller_logs(self): | ||
3736 | 573 | return self.get_logs_by_name('ipcontroller') | ||
3737 | 574 | |||
3738 | 575 | def get_ipcluster_logs(self): | ||
3739 | 576 | return self.get_logs_by_name('ipcluster') | ||
3740 | 577 | |||
3741 | 578 | def get_logs_by_name(self, name='ipcluster'): | ||
3742 | 579 | log_dir = self.cluster_dir_obj.log_dir | ||
3743 | 580 | logs = {} | ||
3744 | 581 | for log in os.listdir(log_dir): | ||
3745 | 582 | if log.startswith(name + '-') and log.endswith('.log'): | ||
3746 | 583 | with open(os.path.join(log_dir, log), 'r') as f: | ||
3747 | 584 | logs[log] = f.read() | ||
3748 | 585 | return logs | ||
3749 | 586 | |||
3750 | 587 | def get_logs(self): | ||
3751 | 588 | d = self.get_ipcluster_logs() | ||
3752 | 589 | d.update(self.get_ipengine_logs()) | ||
3753 | 590 | d.update(self.get_ipcontroller_logs()) | ||
3754 | 591 | return d | ||
3755 | 592 | |||
3756 | 593 | def _handle_start(self, r): | ||
3757 | 594 | self.state = 'running' | ||
3758 | 595 | |||
3759 | 596 | def _handle_stop(self, r): | ||
3760 | 597 | self.state = 'after' | ||
3761 | 598 | |||
3762 | 599 | |||
3763 | 600 | class Cluster(object): | ||
3764 | 601 | |||
3765 | 602 | |||
3766 | 603 | def __init__(self, profile='default', cluster_dir=None, ipython_dir=None, | ||
3767 | 604 | auto_create=False, auto_stop=True): | ||
3768 | 605 | """Create a class to manage an IPython cluster. | ||
3769 | 606 | |||
3770 | 607 | This class calls the :command:`ipcluster` command with the right | ||
3771 | 608 | options to start an IPython cluster. Typically a cluster directory | ||
3772 | 609 | must be created (:command:`ipcluster create`) and configured before | ||
3773 | 610 | using this class. Configuration is done by editing the | ||
3774 | 611 | configuration files in the top level of the cluster directory. | ||
3775 | 612 | |||
3776 | 613 | Parameters | ||
3777 | 614 | ---------- | ||
3778 | 615 | profile : str | ||
3779 | 616 | The name of a cluster directory profile (default="default"). The | ||
3780 | 617 | cluster directory "cluster_<profile>" will be searched for | ||
3781 | 618 | in ``os.getcwd()``, the ipython_dir and then in the directories | ||
3782 | 619 | listed in the :env:`IPCLUSTER_DIR_PATH` environment variable. | ||
3783 | 620 | cluster_dir : str | ||
3784 | 621 | The full path to a cluster directory. This is useful if profiles | ||
3785 | 622 | are not being used. | ||
3786 | 623 | ipython_dir : str | ||
3787 | 624 | The location of the ipython_dir if different from the default. | ||
3788 | 625 | This is used if the cluster directory is being found by profile. | ||
3789 | 626 | auto_create : bool | ||
3790 | 627 | Automatically create the cluster directory it is dones't exist. | ||
3791 | 628 | This will usually only make sense if using a local cluster | ||
3792 | 629 | (default=False). | ||
3793 | 630 | auto_stop : bool | ||
3794 | 631 | Automatically stop the cluster when this instance is garbage | ||
3795 | 632 | collected (default=True). This is useful if you want the cluster | ||
3796 | 633 | to live beyond your current process. There is also an instance | ||
3797 | 634 | attribute ``auto_stop`` to change this behavior. | ||
3798 | 635 | """ | ||
3799 | 636 | self.async_cluster = AsyncCluster( | ||
3800 | 637 | profile, cluster_dir, ipython_dir, auto_create, auto_stop | ||
3801 | 638 | ) | ||
3802 | 639 | self.cluster_dir_obj = self.async_cluster.cluster_dir_obj | ||
3803 | 640 | self.client_connector = None | ||
3804 | 641 | |||
3805 | 642 | def _set_auto_stop(self, value): | ||
3806 | 643 | self.async_cluster.auto_stop = value | ||
3807 | 644 | |||
3808 | 645 | def _get_auto_stop(self): | ||
3809 | 646 | return self.async_cluster.auto_stop | ||
3810 | 647 | |||
3811 | 648 | auto_stop = property(_get_auto_stop, _set_auto_stop) | ||
3812 | 649 | |||
3813 | 650 | @property | ||
3814 | 651 | def location(self): | ||
3815 | 652 | return self.async_cluster.location | ||
3816 | 653 | |||
3817 | 654 | @property | ||
3818 | 655 | def running(self): | ||
3819 | 656 | return self.async_cluster.running | ||
3820 | 657 | |||
3821 | 658 | def start(self, n=2): | ||
3822 | 659 | """Start the IPython cluster with n engines. | ||
3823 | 660 | |||
3824 | 661 | Parameters | ||
3825 | 662 | ---------- | ||
3826 | 663 | n : int | ||
3827 | 664 | The number of engine to start. | ||
3828 | 665 | """ | ||
3829 | 666 | return blockingCallFromThread(self.async_cluster.start, n) | ||
3830 | 667 | |||
3831 | 668 | def stop(self): | ||
3832 | 669 | """Stop the IPython cluster if it is running.""" | ||
3833 | 670 | return blockingCallFromThread(self.async_cluster.stop) | ||
3834 | 671 | |||
3835 | 672 | def get_multiengine_client(self, delay=DELAY, max_tries=MAX_TRIES): | ||
3836 | 673 | """Get the multiengine client for the running cluster. | ||
3837 | 674 | |||
3838 | 675 | This will try to attempt to the controller multiple times. If this | ||
3839 | 676 | fails altogether, try looking at the following: | ||
3840 | 677 | * Make sure the controller is starting properly by looking at its | ||
3841 | 678 | log files. | ||
3842 | 679 | * Make sure the controller is writing its FURL file in the location | ||
3843 | 680 | expected by the client. | ||
3844 | 681 | * Make sure a firewall on the controller's host is not blocking the | ||
3845 | 682 | client from connecting. | ||
3846 | 683 | |||
3847 | 684 | Parameters | ||
3848 | 685 | ---------- | ||
3849 | 686 | delay : float | ||
3850 | 687 | The initial delay between re-connection attempts. Susequent delays | ||
3851 | 688 | get longer according to ``delay[i] = 1.5*delay[i-1]``. | ||
3852 | 689 | max_tries : int | ||
3853 | 690 | The max number of re-connection attempts. | ||
3854 | 691 | """ | ||
3855 | 692 | if self.client_connector is None: | ||
3856 | 693 | self.client_connector = ClientConnector() | ||
3857 | 694 | return self.client_connector.get_multiengine_client( | ||
3858 | 695 | cluster_dir=self.cluster_dir_obj.location, | ||
3859 | 696 | delay=delay, max_tries=max_tries | ||
3860 | 697 | ) | ||
3861 | 698 | |||
3862 | 699 | def get_task_client(self, delay=DELAY, max_tries=MAX_TRIES): | ||
3863 | 700 | """Get the task client for the running cluster. | ||
3864 | 701 | |||
3865 | 702 | This will try to attempt to the controller multiple times. If this | ||
3866 | 703 | fails altogether, try looking at the following: | ||
3867 | 704 | * Make sure the controller is starting properly by looking at its | ||
3868 | 705 | log files. | ||
3869 | 706 | * Make sure the controller is writing its FURL file in the location | ||
3870 | 707 | expected by the client. | ||
3871 | 708 | * Make sure a firewall on the controller's host is not blocking the | ||
3872 | 709 | client from connecting. | ||
3873 | 710 | |||
3874 | 711 | Parameters | ||
3875 | 712 | ---------- | ||
3876 | 713 | delay : float | ||
3877 | 714 | The initial delay between re-connection attempts. Susequent delays | ||
3878 | 715 | get longer according to ``delay[i] = 1.5*delay[i-1]``. | ||
3879 | 716 | max_tries : int | ||
3880 | 717 | The max number of re-connection attempts. | ||
3881 | 718 | """ | ||
3882 | 719 | if self.client_connector is None: | ||
3883 | 720 | self.client_connector = ClientConnector() | ||
3884 | 721 | return self.client_connector.get_task_client( | ||
3885 | 722 | cluster_dir=self.cluster_dir_obj.location, | ||
3886 | 723 | delay=delay, max_tries=max_tries | ||
3887 | 724 | ) | ||
3888 | 725 | |||
3889 | 726 | def __repr__(self): | ||
3890 | 727 | s = "<Cluster(running=%r, location=%s)" % (self.running, self.location) | ||
3891 | 728 | return s | ||
3892 | 729 | |||
3893 | 730 | def get_logs_by_name(self, name='ipcluter'): | ||
3894 | 731 | """Get a dict of logs by process name (ipcluster, ipengine, etc.)""" | ||
3895 | 732 | return self.async_cluster.get_logs_by_name(name) | ||
3896 | 733 | |||
3897 | 734 | def get_ipengine_logs(self): | ||
3898 | 735 | """Get a dict of logs for all engines in this cluster.""" | ||
3899 | 736 | return self.async_cluster.get_ipengine_logs() | ||
3900 | 737 | |||
3901 | 738 | def get_ipcontroller_logs(self): | ||
3902 | 739 | """Get a dict of logs for the controller in this cluster.""" | ||
3903 | 740 | return self.async_cluster.get_ipcontroller_logs() | ||
3904 | 741 | |||
3905 | 742 | def get_ipcluster_logs(self): | ||
3906 | 743 | """Get a dict of the ipcluster logs for this cluster.""" | ||
3907 | 744 | return self.async_cluster.get_ipcluster_logs() | ||
3908 | 745 | |||
3909 | 746 | def get_logs(self): | ||
3910 | 747 | """Get a dict of all logs for this cluster.""" | ||
3911 | 748 | return self.async_cluster.get_logs() | ||
3912 | 749 | |||
3913 | 750 | def _print_logs(self, logs): | ||
3914 | 751 | for k, v in logs.iteritems(): | ||
3915 | 752 | print "===================================" | ||
3916 | 753 | print "Logfile: %s" % k | ||
3917 | 754 | print "===================================" | ||
3918 | 755 | print v | ||
3919 | 756 | |||
3920 | 757 | |||
3921 | 758 | def print_ipengine_logs(self): | ||
3922 | 759 | """Print the ipengine logs for this cluster to stdout.""" | ||
3923 | 760 | self._print_logs(self.get_ipengine_logs()) | ||
3924 | 761 | |||
3925 | 762 | def print_ipcontroller_logs(self): | ||
3926 | 763 | """Print the ipcontroller logs for this cluster to stdout.""" | ||
3927 | 764 | self._print_logs(self.get_ipcontroller_logs()) | ||
3928 | 765 | |||
3929 | 766 | def print_ipcluster_logs(self): | ||
3930 | 767 | """Print the ipcluster logs for this cluster to stdout.""" | ||
3931 | 768 | self._print_logs(self.get_ipcluster_logs()) | ||
3932 | 769 | |||
3933 | 770 | def print_logs(self): | ||
3934 | 771 | """Print all the logs for this cluster to stdout.""" | ||
3935 | 772 | self._print_logs(self.get_logs()) | ||
3936 | 773 | |||
3937 | 151 | 774 | ||
3938 | === added file 'IPython/kernel/clusterdir.py' | |||
3939 | --- IPython/kernel/clusterdir.py 1970-01-01 00:00:00 +0000 | |||
3940 | +++ IPython/kernel/clusterdir.py 2009-11-21 00:11:11 +0000 | |||
3941 | @@ -0,0 +1,475 @@ | |||
3942 | 1 | #!/usr/bin/env python | ||
3943 | 2 | # encoding: utf-8 | ||
3944 | 3 | """ | ||
3945 | 4 | The IPython cluster directory | ||
3946 | 5 | """ | ||
3947 | 6 | |||
3948 | 7 | #----------------------------------------------------------------------------- | ||
3949 | 8 | # Copyright (C) 2008-2009 The IPython Development Team | ||
3950 | 9 | # | ||
3951 | 10 | # Distributed under the terms of the BSD License. The full license is in | ||
3952 | 11 | # the file COPYING, distributed as part of this software. | ||
3953 | 12 | #----------------------------------------------------------------------------- | ||
3954 | 13 | |||
3955 | 14 | #----------------------------------------------------------------------------- | ||
3956 | 15 | # Imports | ||
3957 | 16 | #----------------------------------------------------------------------------- | ||
3958 | 17 | |||
3959 | 18 | from __future__ import with_statement | ||
3960 | 19 | |||
3961 | 20 | import os | ||
3962 | 21 | import shutil | ||
3963 | 22 | import sys | ||
3964 | 23 | |||
3965 | 24 | from twisted.python import log | ||
3966 | 25 | |||
3967 | 26 | from IPython.core import release | ||
3968 | 27 | from IPython.config.loader import PyFileConfigLoader | ||
3969 | 28 | from IPython.core.application import Application | ||
3970 | 29 | from IPython.core.component import Component | ||
3971 | 30 | from IPython.config.loader import ArgParseConfigLoader, NoConfigDefault | ||
3972 | 31 | from IPython.utils.traitlets import Unicode, Bool | ||
3973 | 32 | from IPython.utils import genutils | ||
3974 | 33 | |||
3975 | 34 | #----------------------------------------------------------------------------- | ||
3976 | 35 | # Imports | ||
3977 | 36 | #----------------------------------------------------------------------------- | ||
3978 | 37 | |||
3979 | 38 | |||
3980 | 39 | class ClusterDirError(Exception): | ||
3981 | 40 | pass | ||
3982 | 41 | |||
3983 | 42 | |||
3984 | 43 | class PIDFileError(Exception): | ||
3985 | 44 | pass | ||
3986 | 45 | |||
3987 | 46 | |||
3988 | 47 | class ClusterDir(Component): | ||
3989 | 48 | """An object to manage the cluster directory and its resources. | ||
3990 | 49 | |||
3991 | 50 | The cluster directory is used by :command:`ipcontroller`, | ||
3992 | 51 | :command:`ipcontroller` and :command:`ipcontroller` to manage the | ||
3993 | 52 | configuration, logging and security of these applications. | ||
3994 | 53 | |||
3995 | 54 | This object knows how to find, create and manage these directories. This | ||
3996 | 55 | should be used by any code that want's to handle cluster directories. | ||
3997 | 56 | """ | ||
3998 | 57 | |||
3999 | 58 | security_dir_name = Unicode('security') | ||
4000 | 59 | log_dir_name = Unicode('log') | ||
4001 | 60 | pid_dir_name = Unicode('pid') | ||
4002 | 61 | security_dir = Unicode(u'') | ||
4003 | 62 | log_dir = Unicode(u'') | ||
4004 | 63 | pid_dir = Unicode(u'') | ||
4005 | 64 | location = Unicode(u'') | ||
4006 | 65 | |||
4007 | 66 | def __init__(self, location): | ||
4008 | 67 | super(ClusterDir, self).__init__(None) | ||
4009 | 68 | self.location = location | ||
4010 | 69 | |||
4011 | 70 | def _location_changed(self, name, old, new): | ||
4012 | 71 | if not os.path.isdir(new): | ||
4013 | 72 | os.makedirs(new) | ||
4014 | 73 | self.security_dir = os.path.join(new, self.security_dir_name) | ||
4015 | 74 | self.log_dir = os.path.join(new, self.log_dir_name) | ||
4016 | 75 | self.pid_dir = os.path.join(new, self.pid_dir_name) | ||
4017 | 76 | self.check_dirs() | ||
4018 | 77 | |||
4019 | 78 | def _log_dir_changed(self, name, old, new): | ||
4020 | 79 | self.check_log_dir() | ||
4021 | 80 | |||
4022 | 81 | def check_log_dir(self): | ||
4023 | 82 | if not os.path.isdir(self.log_dir): | ||
4024 | 83 | os.mkdir(self.log_dir) | ||
4025 | 84 | |||
4026 | 85 | def _security_dir_changed(self, name, old, new): | ||
4027 | 86 | self.check_security_dir() | ||
4028 | 87 | |||
4029 | 88 | def check_security_dir(self): | ||
4030 | 89 | if not os.path.isdir(self.security_dir): | ||
4031 | 90 | os.mkdir(self.security_dir, 0700) | ||
4032 | 91 | os.chmod(self.security_dir, 0700) | ||
4033 | 92 | |||
4034 | 93 | def _pid_dir_changed(self, name, old, new): | ||
4035 | 94 | self.check_pid_dir() | ||
4036 | 95 | |||
4037 | 96 | def check_pid_dir(self): | ||
4038 | 97 | if not os.path.isdir(self.pid_dir): | ||
4039 | 98 | os.mkdir(self.pid_dir, 0700) | ||
4040 | 99 | os.chmod(self.pid_dir, 0700) | ||
4041 | 100 | |||
4042 | 101 | def check_dirs(self): | ||
4043 | 102 | self.check_security_dir() | ||
4044 | 103 | self.check_log_dir() | ||
4045 | 104 | self.check_pid_dir() | ||
4046 | 105 | |||
4047 | 106 | def load_config_file(self, filename): | ||
4048 | 107 | """Load a config file from the top level of the cluster dir. | ||
4049 | 108 | |||
4050 | 109 | Parameters | ||
4051 | 110 | ---------- | ||
4052 | 111 | filename : unicode or str | ||
4053 | 112 | The filename only of the config file that must be located in | ||
4054 | 113 | the top-level of the cluster directory. | ||
4055 | 114 | """ | ||
4056 | 115 | loader = PyFileConfigLoader(filename, self.location) | ||
4057 | 116 | return loader.load_config() | ||
4058 | 117 | |||
4059 | 118 | def copy_config_file(self, config_file, path=None, overwrite=False): | ||
4060 | 119 | """Copy a default config file into the active cluster directory. | ||
4061 | 120 | |||
4062 | 121 | Default configuration files are kept in :mod:`IPython.config.default`. | ||
4063 | 122 | This function moves these from that location to the working cluster | ||
4064 | 123 | directory. | ||
4065 | 124 | """ | ||
4066 | 125 | if path is None: | ||
4067 | 126 | import IPython.config.default | ||
4068 | 127 | path = IPython.config.default.__file__.split(os.path.sep)[:-1] | ||
4069 | 128 | path = os.path.sep.join(path) | ||
4070 | 129 | src = os.path.join(path, config_file) | ||
4071 | 130 | dst = os.path.join(self.location, config_file) | ||
4072 | 131 | if not os.path.isfile(dst) or overwrite: | ||
4073 | 132 | shutil.copy(src, dst) | ||
4074 | 133 | |||
4075 | 134 | def copy_all_config_files(self, path=None, overwrite=False): | ||
4076 | 135 | """Copy all config files into the active cluster directory.""" | ||
4077 | 136 | for f in [u'ipcontroller_config.py', u'ipengine_config.py', | ||
4078 | 137 | u'ipcluster_config.py']: | ||
4079 | 138 | self.copy_config_file(f, path=path, overwrite=overwrite) | ||
4080 | 139 | |||
4081 | 140 | @classmethod | ||
4082 | 141 | def create_cluster_dir(csl, cluster_dir): | ||
4083 | 142 | """Create a new cluster directory given a full path. | ||
4084 | 143 | |||
4085 | 144 | Parameters | ||
4086 | 145 | ---------- | ||
4087 | 146 | cluster_dir : str | ||
4088 | 147 | The full path to the cluster directory. If it does exist, it will | ||
4089 | 148 | be used. If not, it will be created. | ||
4090 | 149 | """ | ||
4091 | 150 | return ClusterDir(cluster_dir) | ||
4092 | 151 | |||
4093 | 152 | @classmethod | ||
4094 | 153 | def create_cluster_dir_by_profile(cls, path, profile=u'default'): | ||
4095 | 154 | """Create a cluster dir by profile name and path. | ||
4096 | 155 | |||
4097 | 156 | Parameters | ||
4098 | 157 | ---------- | ||
4099 | 158 | path : str | ||
4100 | 159 | The path (directory) to put the cluster directory in. | ||
4101 | 160 | profile : str | ||
4102 | 161 | The name of the profile. The name of the cluster directory will | ||
4103 | 162 | be "cluster_<profile>". | ||
4104 | 163 | """ | ||
4105 | 164 | if not os.path.isdir(path): | ||
4106 | 165 | raise ClusterDirError('Directory not found: %s' % path) | ||
4107 | 166 | cluster_dir = os.path.join(path, u'cluster_' + profile) | ||
4108 | 167 | return ClusterDir(cluster_dir) | ||
4109 | 168 | |||
4110 | 169 | @classmethod | ||
4111 | 170 | def find_cluster_dir_by_profile(cls, ipython_dir, profile=u'default'): | ||
4112 | 171 | """Find an existing cluster dir by profile name, return its ClusterDir. | ||
4113 | 172 | |||
4114 | 173 | This searches through a sequence of paths for a cluster dir. If it | ||
4115 | 174 | is not found, a :class:`ClusterDirError` exception will be raised. | ||
4116 | 175 | |||
4117 | 176 | The search path algorithm is: | ||
4118 | 177 | 1. ``os.getcwd()`` | ||
4119 | 178 | 2. ``ipython_dir`` | ||
4120 | 179 | 3. The directories found in the ":" separated | ||
4121 | 180 | :env:`IPCLUSTER_DIR_PATH` environment variable. | ||
4122 | 181 | |||
4123 | 182 | Parameters | ||
4124 | 183 | ---------- | ||
4125 | 184 | ipython_dir : unicode or str | ||
4126 | 185 | The IPython directory to use. | ||
4127 | 186 | profile : unicode or str | ||
4128 | 187 | The name of the profile. The name of the cluster directory | ||
4129 | 188 | will be "cluster_<profile>". | ||
4130 | 189 | """ | ||
4131 | 190 | dirname = u'cluster_' + profile | ||
4132 | 191 | cluster_dir_paths = os.environ.get('IPCLUSTER_DIR_PATH','') | ||
4133 | 192 | if cluster_dir_paths: | ||
4134 | 193 | cluster_dir_paths = cluster_dir_paths.split(':') | ||
4135 | 194 | else: | ||
4136 | 195 | cluster_dir_paths = [] | ||
4137 | 196 | paths = [os.getcwd(), ipython_dir] + cluster_dir_paths | ||
4138 | 197 | for p in paths: | ||
4139 | 198 | cluster_dir = os.path.join(p, dirname) | ||
4140 | 199 | if os.path.isdir(cluster_dir): | ||
4141 | 200 | return ClusterDir(cluster_dir) | ||
4142 | 201 | else: | ||
4143 | 202 | raise ClusterDirError('Cluster directory not found in paths: %s' % dirname) | ||
4144 | 203 | |||
4145 | 204 | @classmethod | ||
4146 | 205 | def find_cluster_dir(cls, cluster_dir): | ||
4147 | 206 | """Find/create a cluster dir and return its ClusterDir. | ||
4148 | 207 | |||
4149 | 208 | This will create the cluster directory if it doesn't exist. | ||
4150 | 209 | |||
4151 | 210 | Parameters | ||
4152 | 211 | ---------- | ||
4153 | 212 | cluster_dir : unicode or str | ||
4154 | 213 | The path of the cluster directory. This is expanded using | ||
4155 | 214 | :func:`IPython.utils.genutils.expand_path`. | ||
4156 | 215 | """ | ||
4157 | 216 | cluster_dir = genutils.expand_path(cluster_dir) | ||
4158 | 217 | if not os.path.isdir(cluster_dir): | ||
4159 | 218 | raise ClusterDirError('Cluster directory not found: %s' % cluster_dir) | ||
4160 | 219 | return ClusterDir(cluster_dir) | ||
4161 | 220 | |||
4162 | 221 | |||
4163 | 222 | class AppWithClusterDirArgParseConfigLoader(ArgParseConfigLoader): | ||
4164 | 223 | """Default command line options for IPython cluster applications.""" | ||
4165 | 224 | |||
4166 | 225 | def _add_other_arguments(self): | ||
4167 | 226 | self.parser.add_argument('--ipython-dir', | ||
4168 | 227 | dest='Global.ipython_dir',type=unicode, | ||
4169 | 228 | help='Set to override default location of Global.ipython_dir.', | ||
4170 | 229 | default=NoConfigDefault, | ||
4171 | 230 | metavar='Global.ipython_dir' | ||
4172 | 231 | ) | ||
4173 | 232 | self.parser.add_argument('-p', '--profile', | ||
4174 | 233 | dest='Global.profile',type=unicode, | ||
4175 | 234 | help='The string name of the profile to be used. This determines ' | ||
4176 | 235 | 'the name of the cluster dir as: cluster_<profile>. The default profile ' | ||
4177 | 236 | 'is named "default". The cluster directory is resolve this way ' | ||
4178 | 237 | 'if the --cluster-dir option is not used.', | ||
4179 | 238 | default=NoConfigDefault, | ||
4180 | 239 | metavar='Global.profile' | ||
4181 | 240 | ) | ||
4182 | 241 | self.parser.add_argument('--log-level', | ||
4183 | 242 | dest="Global.log_level",type=int, | ||
4184 | 243 | help='Set the log level (0,10,20,30,40,50). Default is 30.', | ||
4185 | 244 | default=NoConfigDefault, | ||
4186 | 245 | metavar="Global.log_level" | ||
4187 | 246 | ) | ||
4188 | 247 | self.parser.add_argument('--cluster-dir', | ||
4189 | 248 | dest='Global.cluster_dir',type=unicode, | ||
4190 | 249 | help='Set the cluster dir. This overrides the logic used by the ' | ||
4191 | 250 | '--profile option.', | ||
4192 | 251 | default=NoConfigDefault, | ||
4193 | 252 | metavar='Global.cluster_dir' | ||
4194 | 253 | ), | ||
4195 | 254 | self.parser.add_argument('--work-dir', | ||
4196 | 255 | dest='Global.work_dir',type=unicode, | ||
4197 | 256 | help='Set the working dir for the process.', | ||
4198 | 257 | default=NoConfigDefault, | ||
4199 | 258 | metavar='Global.work_dir' | ||
4200 | 259 | ) | ||
4201 | 260 | self.parser.add_argument('--clean-logs', | ||
4202 | 261 | dest='Global.clean_logs', action='store_true', | ||
4203 | 262 | help='Delete old log flies before starting.', | ||
4204 | 263 | default=NoConfigDefault | ||
4205 | 264 | ) | ||
4206 | 265 | self.parser.add_argument('--no-clean-logs', | ||
4207 | 266 | dest='Global.clean_logs', action='store_false', | ||
4208 | 267 | help="Don't Delete old log flies before starting.", | ||
4209 | 268 | default=NoConfigDefault | ||
4210 | 269 | ) | ||
4211 | 270 | |||
4212 | 271 | class ApplicationWithClusterDir(Application): | ||
4213 | 272 | """An application that puts everything into a cluster directory. | ||
4214 | 273 | |||
4215 | 274 | Instead of looking for things in the ipython_dir, this type of application | ||
4216 | 275 | will use its own private directory called the "cluster directory" | ||
4217 | 276 | for things like config files, log files, etc. | ||
4218 | 277 | |||
4219 | 278 | The cluster directory is resolved as follows: | ||
4220 | 279 | |||
4221 | 280 | * If the ``--cluster-dir`` option is given, it is used. | ||
4222 | 281 | * If ``--cluster-dir`` is not given, the application directory is | ||
4223 | 282 | resolve using the profile name as ``cluster_<profile>``. The search | ||
4224 | 283 | path for this directory is then i) cwd if it is found there | ||
4225 | 284 | and ii) in ipython_dir otherwise. | ||
4226 | 285 | |||
4227 | 286 | The config file for the application is to be put in the cluster | ||
4228 | 287 | dir and named the value of the ``config_file_name`` class attribute. | ||
4229 | 288 | """ | ||
4230 | 289 | |||
4231 | 290 | auto_create_cluster_dir = True | ||
4232 | 291 | |||
4233 | 292 | def create_default_config(self): | ||
4234 | 293 | super(ApplicationWithClusterDir, self).create_default_config() | ||
4235 | 294 | self.default_config.Global.profile = u'default' | ||
4236 | 295 | self.default_config.Global.cluster_dir = u'' | ||
4237 | 296 | self.default_config.Global.work_dir = os.getcwd() | ||
4238 | 297 | self.default_config.Global.log_to_file = False | ||
4239 | 298 | self.default_config.Global.clean_logs = False | ||
4240 | 299 | |||
4241 | 300 | def create_command_line_config(self): | ||
4242 | 301 | """Create and return a command line config loader.""" | ||
4243 | 302 | return AppWithClusterDirArgParseConfigLoader( | ||
4244 | 303 | description=self.description, | ||
4245 | 304 | version=release.version | ||
4246 | 305 | ) | ||
4247 | 306 | |||
4248 | 307 | def find_resources(self): | ||
4249 | 308 | """This resolves the cluster directory. | ||
4250 | 309 | |||
4251 | 310 | This tries to find the cluster directory and if successful, it will | ||
4252 | 311 | have done: | ||
4253 | 312 | * Sets ``self.cluster_dir_obj`` to the :class:`ClusterDir` object for | ||
4254 | 313 | the application. | ||
4255 | 314 | * Sets ``self.cluster_dir`` attribute of the application and config | ||
4256 | 315 | objects. | ||
4257 | 316 | |||
4258 | 317 | The algorithm used for this is as follows: | ||
4259 | 318 | 1. Try ``Global.cluster_dir``. | ||
4260 | 319 | 2. Try using ``Global.profile``. | ||
4261 | 320 | 3. If both of these fail and ``self.auto_create_cluster_dir`` is | ||
4262 | 321 | ``True``, then create the new cluster dir in the IPython directory. | ||
4263 | 322 | 4. If all fails, then raise :class:`ClusterDirError`. | ||
4264 | 323 | """ | ||
4265 | 324 | |||
4266 | 325 | try: | ||
4267 | 326 | cluster_dir = self.command_line_config.Global.cluster_dir | ||
4268 | 327 | except AttributeError: | ||
4269 | 328 | cluster_dir = self.default_config.Global.cluster_dir | ||
4270 | 329 | cluster_dir = genutils.expand_path(cluster_dir) | ||
4271 | 330 | try: | ||
4272 | 331 | self.cluster_dir_obj = ClusterDir.find_cluster_dir(cluster_dir) | ||
4273 | 332 | except ClusterDirError: | ||
4274 | 333 | pass | ||
4275 | 334 | else: | ||
4276 | 335 | self.log.info('Using existing cluster dir: %s' % \ | ||
4277 | 336 | self.cluster_dir_obj.location | ||
4278 | 337 | ) | ||
4279 | 338 | self.finish_cluster_dir() | ||
4280 | 339 | return | ||
4281 | 340 | |||
4282 | 341 | try: | ||
4283 | 342 | self.profile = self.command_line_config.Global.profile | ||
4284 | 343 | except AttributeError: | ||
4285 | 344 | self.profile = self.default_config.Global.profile | ||
4286 | 345 | try: | ||
4287 | 346 | self.cluster_dir_obj = ClusterDir.find_cluster_dir_by_profile( | ||
4288 | 347 | self.ipython_dir, self.profile) | ||
4289 | 348 | except ClusterDirError: | ||
4290 | 349 | pass | ||
4291 | 350 | else: | ||
4292 | 351 | self.log.info('Using existing cluster dir: %s' % \ | ||
4293 | 352 | self.cluster_dir_obj.location | ||
4294 | 353 | ) | ||
4295 | 354 | self.finish_cluster_dir() | ||
4296 | 355 | return | ||
4297 | 356 | |||
4298 | 357 | if self.auto_create_cluster_dir: | ||
4299 | 358 | self.cluster_dir_obj = ClusterDir.create_cluster_dir_by_profile( | ||
4300 | 359 | self.ipython_dir, self.profile | ||
4301 | 360 | ) | ||
4302 | 361 | self.log.info('Creating new cluster dir: %s' % \ | ||
4303 | 362 | self.cluster_dir_obj.location | ||
4304 | 363 | ) | ||
4305 | 364 | self.finish_cluster_dir() | ||
4306 | 365 | else: | ||
4307 | 366 | raise ClusterDirError('Could not find a valid cluster directory.') | ||
4308 | 367 | |||
4309 | 368 | def finish_cluster_dir(self): | ||
4310 | 369 | # Set the cluster directory | ||
4311 | 370 | self.cluster_dir = self.cluster_dir_obj.location | ||
4312 | 371 | |||
4313 | 372 | # These have to be set because they could be different from the one | ||
4314 | 373 | # that we just computed. Because command line has the highest | ||
4315 | 374 | # priority, this will always end up in the master_config. | ||
4316 | 375 | self.default_config.Global.cluster_dir = self.cluster_dir | ||
4317 | 376 | self.command_line_config.Global.cluster_dir = self.cluster_dir | ||
4318 | 377 | |||
4319 | 378 | # Set the search path to the cluster directory | ||
4320 | 379 | self.config_file_paths = (self.cluster_dir,) | ||
4321 | 380 | |||
4322 | 381 | def find_config_file_name(self): | ||
4323 | 382 | """Find the config file name for this application.""" | ||
4324 | 383 | # For this type of Application it should be set as a class attribute. | ||
4325 | 384 | if not hasattr(self, 'config_file_name'): | ||
4326 | 385 | self.log.critical("No config filename found") | ||
4327 | 386 | |||
4328 | 387 | def find_config_file_paths(self): | ||
4329 | 388 | # Set the search path to the cluster directory | ||
4330 | 389 | self.config_file_paths = (self.cluster_dir,) | ||
4331 | 390 | |||
4332 | 391 | def pre_construct(self): | ||
4333 | 392 | # The log and security dirs were set earlier, but here we put them | ||
4334 | 393 | # into the config and log them. | ||
4335 | 394 | config = self.master_config | ||
4336 | 395 | sdir = self.cluster_dir_obj.security_dir | ||
4337 | 396 | self.security_dir = config.Global.security_dir = sdir | ||
4338 | 397 | ldir = self.cluster_dir_obj.log_dir | ||
4339 | 398 | self.log_dir = config.Global.log_dir = ldir | ||
4340 | 399 | pdir = self.cluster_dir_obj.pid_dir | ||
4341 | 400 | self.pid_dir = config.Global.pid_dir = pdir | ||
4342 | 401 | self.log.info("Cluster directory set to: %s" % self.cluster_dir) | ||
4343 | 402 | config.Global.work_dir = unicode(genutils.expand_path(config.Global.work_dir)) | ||
4344 | 403 | # Change to the working directory. We do this just before construct | ||
4345 | 404 | # is called so all the components there have the right working dir. | ||
4346 | 405 | self.to_work_dir() | ||
4347 | 406 | |||
4348 | 407 | def to_work_dir(self): | ||
4349 | 408 | wd = self.master_config.Global.work_dir | ||
4350 | 409 | if unicode(wd) != unicode(os.getcwd()): | ||
4351 | 410 | os.chdir(wd) | ||
4352 | 411 | self.log.info("Changing to working dir: %s" % wd) | ||
4353 | 412 | |||
4354 | 413 | def start_logging(self): | ||
4355 | 414 | # Remove old log files | ||
4356 | 415 | if self.master_config.Global.clean_logs: | ||
4357 | 416 | log_dir = self.master_config.Global.log_dir | ||
4358 | 417 | for f in os.listdir(log_dir): | ||
4359 | 418 | if f.startswith(self.name + u'-') and f.endswith('.log'): | ||
4360 | 419 | os.remove(os.path.join(log_dir, f)) | ||
4361 | 420 | # Start logging to the new log file | ||
4362 | 421 | if self.master_config.Global.log_to_file: | ||
4363 | 422 | log_filename = self.name + u'-' + str(os.getpid()) + u'.log' | ||
4364 | 423 | logfile = os.path.join(self.log_dir, log_filename) | ||
4365 | 424 | open_log_file = open(logfile, 'w') | ||
4366 | 425 | else: | ||
4367 | 426 | open_log_file = sys.stdout | ||
4368 | 427 | log.startLogging(open_log_file) | ||
4369 | 428 | |||
4370 | 429 | def write_pid_file(self, overwrite=False): | ||
4371 | 430 | """Create a .pid file in the pid_dir with my pid. | ||
4372 | 431 | |||
4373 | 432 | This must be called after pre_construct, which sets `self.pid_dir`. | ||
4374 | 433 | This raises :exc:`PIDFileError` if the pid file exists already. | ||
4375 | 434 | """ | ||
4376 | 435 | pid_file = os.path.join(self.pid_dir, self.name + u'.pid') | ||
4377 | 436 | if os.path.isfile(pid_file): | ||
4378 | 437 | pid = self.get_pid_from_file() | ||
4379 | 438 | if not overwrite: | ||
4380 | 439 | raise PIDFileError( | ||
4381 | 440 | 'The pid file [%s] already exists. \nThis could mean that this ' | ||
4382 | 441 | 'server is already running with [pid=%s].' % (pid_file, pid) | ||
4383 | 442 | ) | ||
4384 | 443 | with open(pid_file, 'w') as f: | ||
4385 | 444 | self.log.info("Creating pid file: %s" % pid_file) | ||
4386 | 445 | f.write(repr(os.getpid())+'\n') | ||
4387 | 446 | |||
4388 | 447 | def remove_pid_file(self): | ||
4389 | 448 | """Remove the pid file. | ||
4390 | 449 | |||
4391 | 450 | This should be called at shutdown by registering a callback with | ||
4392 | 451 | :func:`reactor.addSystemEventTrigger`. This needs to return | ||
4393 | 452 | ``None``. | ||
4394 | 453 | """ | ||
4395 | 454 | pid_file = os.path.join(self.pid_dir, self.name + u'.pid') | ||
4396 | 455 | if os.path.isfile(pid_file): | ||
4397 | 456 | try: | ||
4398 | 457 | self.log.info("Removing pid file: %s" % pid_file) | ||
4399 | 458 | os.remove(pid_file) | ||
4400 | 459 | except: | ||
4401 | 460 | self.log.warn("Error removing the pid file: %s" % pid_file) | ||
4402 | 461 | |||
4403 | 462 | def get_pid_from_file(self): | ||
4404 | 463 | """Get the pid from the pid file. | ||
4405 | 464 | |||
4406 | 465 | If the pid file doesn't exist a :exc:`PIDFileError` is raised. | ||
4407 | 466 | """ | ||
4408 | 467 | pid_file = os.path.join(self.pid_dir, self.name + u'.pid') | ||
4409 | 468 | if os.path.isfile(pid_file): | ||
4410 | 469 | with open(pid_file, 'r') as f: | ||
4411 | 470 | pid = int(f.read().strip()) | ||
4412 | 471 | return pid | ||
4413 | 472 | else: | ||
4414 | 473 | raise PIDFileError('pid file not found: %s' % pid_file) | ||
4415 | 474 | |||
4416 | 475 | |||
4417 | 0 | 476 | ||
4418 | === removed directory 'IPython/kernel/config' | |||
4419 | === removed file 'IPython/kernel/config/__init__.py' | |||
4420 | --- IPython/kernel/config/__init__.py 2009-07-02 02:49:32 +0000 | |||
4421 | +++ IPython/kernel/config/__init__.py 1970-01-01 00:00:00 +0000 | |||
4422 | @@ -1,126 +0,0 @@ | |||
4423 | 1 | # encoding: utf-8 | ||
4424 | 2 | |||
4425 | 3 | """Default kernel configuration.""" | ||
4426 | 4 | |||
4427 | 5 | __docformat__ = "restructuredtext en" | ||
4428 | 6 | |||
4429 | 7 | #------------------------------------------------------------------------------- | ||
4430 | 8 | # Copyright (C) 2008 The IPython Development Team | ||
4431 | 9 | # | ||
4432 | 10 | # Distributed under the terms of the BSD License. The full license is in | ||
4433 | 11 | # the file COPYING, distributed as part of this software. | ||
4434 | 12 | #------------------------------------------------------------------------------- | ||
4435 | 13 | |||
4436 | 14 | #------------------------------------------------------------------------------- | ||
4437 | 15 | # Imports | ||
4438 | 16 | #------------------------------------------------------------------------------- | ||
4439 | 17 | |||
4440 | 18 | import os, sys | ||
4441 | 19 | from os.path import join as pjoin | ||
4442 | 20 | |||
4443 | 21 | from IPython.external.configobj import ConfigObj | ||
4444 | 22 | from IPython.config.api import ConfigObjManager | ||
4445 | 23 | from IPython.utils.genutils import get_ipython_dir, get_security_dir | ||
4446 | 24 | |||
4447 | 25 | default_kernel_config = ConfigObj() | ||
4448 | 26 | |||
4449 | 27 | # This will raise OSError if ipythondir doesn't exist. | ||
4450 | 28 | security_dir = get_security_dir() | ||
4451 | 29 | |||
4452 | 30 | #------------------------------------------------------------------------------- | ||
4453 | 31 | # Engine Configuration | ||
4454 | 32 | #------------------------------------------------------------------------------- | ||
4455 | 33 | |||
4456 | 34 | engine_config = dict( | ||
4457 | 35 | logfile = '', # Empty means log to stdout | ||
4458 | 36 | furl_file = pjoin(security_dir, 'ipcontroller-engine.furl') | ||
4459 | 37 | ) | ||
4460 | 38 | |||
4461 | 39 | #------------------------------------------------------------------------------- | ||
4462 | 40 | # MPI Configuration | ||
4463 | 41 | #------------------------------------------------------------------------------- | ||
4464 | 42 | |||
4465 | 43 | mpi_config = dict( | ||
4466 | 44 | mpi4py = """from mpi4py import MPI as mpi | ||
4467 | 45 | mpi.size = mpi.COMM_WORLD.Get_size() | ||
4468 | 46 | mpi.rank = mpi.COMM_WORLD.Get_rank() | ||
4469 | 47 | """, | ||
4470 | 48 | pytrilinos = """from PyTrilinos import Epetra | ||
4471 | 49 | class SimpleStruct: | ||
4472 | 50 | pass | ||
4473 | 51 | mpi = SimpleStruct() | ||
4474 | 52 | mpi.rank = 0 | ||
4475 | 53 | mpi.size = 0 | ||
4476 | 54 | """, | ||
4477 | 55 | default = '' | ||
4478 | 56 | ) | ||
4479 | 57 | |||
4480 | 58 | #------------------------------------------------------------------------------- | ||
4481 | 59 | # Controller Configuration | ||
4482 | 60 | #------------------------------------------------------------------------------- | ||
4483 | 61 | |||
4484 | 62 | controller_config = dict( | ||
4485 | 63 | |||
4486 | 64 | logfile = '', # Empty means log to stdout | ||
4487 | 65 | import_statement = '', | ||
4488 | 66 | reuse_furls = False, # If False, old furl files are deleted | ||
4489 | 67 | |||
4490 | 68 | engine_tub = dict( | ||
4491 | 69 | ip = '', # Empty string means all interfaces | ||
4492 | 70 | port = 0, # 0 means pick a port for me | ||
4493 | 71 | location = '', # Empty string means try to set automatically | ||
4494 | 72 | secure = True, | ||
4495 | 73 | cert_file = pjoin(security_dir, 'ipcontroller-engine.pem'), | ||
4496 | 74 | ), | ||
4497 | 75 | engine_fc_interface = 'IPython.kernel.enginefc.IFCControllerBase', | ||
4498 | 76 | engine_furl_file = pjoin(security_dir, 'ipcontroller-engine.furl'), | ||
4499 | 77 | |||
4500 | 78 | controller_interfaces = dict( | ||
4501 | 79 | # multiengine = dict( | ||
4502 | 80 | # controller_interface = 'IPython.kernel.multiengine.IMultiEngine', | ||
4503 | 81 | # fc_interface = 'IPython.kernel.multienginefc.IFCMultiEngine', | ||
4504 | 82 | # furl_file = 'ipcontroller-mec.furl' | ||
4505 | 83 | # ), | ||
4506 | 84 | task = dict( | ||
4507 | 85 | controller_interface = 'IPython.kernel.task.ITaskController', | ||
4508 | 86 | fc_interface = 'IPython.kernel.taskfc.IFCTaskController', | ||
4509 | 87 | furl_file = pjoin(security_dir, 'ipcontroller-tc.furl') | ||
4510 | 88 | ), | ||
4511 | 89 | multiengine = dict( | ||
4512 | 90 | controller_interface = 'IPython.kernel.multiengine.IMultiEngine', | ||
4513 | 91 | fc_interface = 'IPython.kernel.multienginefc.IFCSynchronousMultiEngine', | ||
4514 | 92 | furl_file = pjoin(security_dir, 'ipcontroller-mec.furl') | ||
4515 | 93 | ) | ||
4516 | 94 | ), | ||
4517 | 95 | |||
4518 | 96 | client_tub = dict( | ||
4519 | 97 | ip = '', # Empty string means all interfaces | ||
4520 | 98 | port = 0, # 0 means pick a port for me | ||
4521 | 99 | location = '', # Empty string means try to set automatically | ||
4522 | 100 | secure = True, | ||
4523 | 101 | cert_file = pjoin(security_dir, 'ipcontroller-client.pem') | ||
4524 | 102 | ) | ||
4525 | 103 | ) | ||
4526 | 104 | |||
4527 | 105 | #------------------------------------------------------------------------------- | ||
4528 | 106 | # Client Configuration | ||
4529 | 107 | #------------------------------------------------------------------------------- | ||
4530 | 108 | |||
4531 | 109 | client_config = dict( | ||
4532 | 110 | client_interfaces = dict( | ||
4533 | 111 | task = dict( | ||
4534 | 112 | furl_file = pjoin(security_dir, 'ipcontroller-tc.furl') | ||
4535 | 113 | ), | ||
4536 | 114 | multiengine = dict( | ||
4537 | 115 | furl_file = pjoin(security_dir, 'ipcontroller-mec.furl') | ||
4538 | 116 | ) | ||
4539 | 117 | ) | ||
4540 | 118 | ) | ||
4541 | 119 | |||
4542 | 120 | default_kernel_config['engine'] = engine_config | ||
4543 | 121 | default_kernel_config['mpi'] = mpi_config | ||
4544 | 122 | default_kernel_config['controller'] = controller_config | ||
4545 | 123 | default_kernel_config['client'] = client_config | ||
4546 | 124 | |||
4547 | 125 | |||
4548 | 126 | config_manager = ConfigObjManager(default_kernel_config, 'IPython.kernel.ini') | ||
4549 | 127 | \ No newline at end of file | 0 | \ No newline at end of file |
4550 | 128 | 1 | ||
4551 | === added file 'IPython/kernel/configobjfactory.py' | |||
4552 | --- IPython/kernel/configobjfactory.py 1970-01-01 00:00:00 +0000 | |||
4553 | +++ IPython/kernel/configobjfactory.py 2009-11-21 00:11:11 +0000 | |||
4554 | @@ -0,0 +1,79 @@ | |||
4555 | 1 | #!/usr/bin/env python | ||
4556 | 2 | # encoding: utf-8 | ||
4557 | 3 | """ | ||
4558 | 4 | A class for creating a Twisted service that is configured using IPython's | ||
4559 | 5 | configuration system. | ||
4560 | 6 | """ | ||
4561 | 7 | |||
4562 | 8 | #----------------------------------------------------------------------------- | ||
4563 | 9 | # Copyright (C) 2008-2009 The IPython Development Team | ||
4564 | 10 | # | ||
4565 | 11 | # Distributed under the terms of the BSD License. The full license is in | ||
4566 | 12 | # the file COPYING, distributed as part of this software. | ||
4567 | 13 | #----------------------------------------------------------------------------- | ||
4568 | 14 | |||
4569 | 15 | #----------------------------------------------------------------------------- | ||
4570 | 16 | # Imports | ||
4571 | 17 | #----------------------------------------------------------------------------- | ||
4572 | 18 | |||
4573 | 19 | import zope.interface as zi | ||
4574 | 20 | |||
4575 | 21 | from IPython.core.component import Component | ||
4576 | 22 | |||
4577 | 23 | #----------------------------------------------------------------------------- | ||
4578 | 24 | # Code | ||
4579 | 25 | #----------------------------------------------------------------------------- | ||
4580 | 26 | |||
4581 | 27 | |||
4582 | 28 | class IConfiguredObjectFactory(zi.Interface): | ||
4583 | 29 | """I am a component that creates a configured object. | ||
4584 | 30 | |||
4585 | 31 | This class is useful if you want to configure a class that is not a | ||
4586 | 32 | subclass of :class:`IPython.core.component.Component`. | ||
4587 | 33 | """ | ||
4588 | 34 | |||
4589 | 35 | def __init__(config): | ||
4590 | 36 | """Get ready to configure the object using config.""" | ||
4591 | 37 | |||
4592 | 38 | def create(): | ||
4593 | 39 | """Return an instance of the configured object.""" | ||
4594 | 40 | |||
4595 | 41 | |||
4596 | 42 | class ConfiguredObjectFactory(Component): | ||
4597 | 43 | |||
4598 | 44 | zi.implements(IConfiguredObjectFactory) | ||
4599 | 45 | |||
4600 | 46 | def __init__(self, config): | ||
4601 | 47 | super(ConfiguredObjectFactory, self).__init__(None, config=config) | ||
4602 | 48 | |||
4603 | 49 | def create(self): | ||
4604 | 50 | raise NotImplementedError('create must be implemented in a subclass') | ||
4605 | 51 | |||
4606 | 52 | |||
4607 | 53 | class IAdaptedConfiguredObjectFactory(zi.Interface): | ||
4608 | 54 | """I am a component that adapts and configures an object. | ||
4609 | 55 | |||
4610 | 56 | This class is useful if you have the adapt an instance and configure it. | ||
4611 | 57 | """ | ||
4612 | 58 | |||
4613 | 59 | def __init__(config, adaptee=None): | ||
4614 | 60 | """Get ready to adapt adaptee and then configure it using config.""" | ||
4615 | 61 | |||
4616 | 62 | def create(): | ||
4617 | 63 | """Return an instance of the adapted and configured object.""" | ||
4618 | 64 | |||
4619 | 65 | |||
4620 | 66 | class AdaptedConfiguredObjectFactory(Component): | ||
4621 | 67 | |||
4622 | 68 | # zi.implements(IAdaptedConfiguredObjectFactory) | ||
4623 | 69 | |||
4624 | 70 | def __init__(self, config, adaptee): | ||
4625 | 71 | |||
4626 | 72 | # print "config pre:", config | ||
4627 | 73 | super(AdaptedConfiguredObjectFactory, self).__init__(None, config=config) | ||
4628 | 74 | |||
4629 | 75 | # print "config post:", config | ||
4630 | 76 | self.adaptee = adaptee | ||
4631 | 77 | |||
4632 | 78 | def create(self): | ||
4633 | 79 | raise NotImplementedError('create must be implemented in a subclass') | ||
4634 | 0 | \ No newline at end of file | 80 | \ No newline at end of file |
4635 | 1 | 81 | ||
4636 | === removed directory 'IPython/kernel/core/config' | |||
4637 | === removed file 'IPython/kernel/core/config/__init__.py' | |||
4638 | --- IPython/kernel/core/config/__init__.py 2008-06-06 19:41:55 +0000 | |||
4639 | +++ IPython/kernel/core/config/__init__.py 1970-01-01 00:00:00 +0000 | |||
4640 | @@ -1,25 +0,0 @@ | |||
4641 | 1 | # encoding: utf-8 | ||
4642 | 2 | |||
4643 | 3 | __docformat__ = "restructuredtext en" | ||
4644 | 4 | |||
4645 | 5 | #------------------------------------------------------------------------------- | ||
4646 | 6 | # Copyright (C) 2008 The IPython Development Team | ||
4647 | 7 | # | ||
4648 | 8 | # Distributed under the terms of the BSD License. The full license is in | ||
4649 | 9 | # the file COPYING, distributed as part of this software. | ||
4650 | 10 | #------------------------------------------------------------------------------- | ||
4651 | 11 | |||
4652 | 12 | #------------------------------------------------------------------------------- | ||
4653 | 13 | # Imports | ||
4654 | 14 | #------------------------------------------------------------------------------- | ||
4655 | 15 | |||
4656 | 16 | from IPython.external.configobj import ConfigObj | ||
4657 | 17 | from IPython.config.api import ConfigObjManager | ||
4658 | 18 | |||
4659 | 19 | default_core_config = ConfigObj() | ||
4660 | 20 | default_core_config['shell'] = dict( | ||
4661 | 21 | shell_class = 'IPython.kernel.core.interpreter.Interpreter', | ||
4662 | 22 | import_statement = '' | ||
4663 | 23 | ) | ||
4664 | 24 | |||
4665 | 25 | config_manager = ConfigObjManager(default_core_config, 'IPython.kernel.core.ini') | ||
4666 | 26 | \ No newline at end of file | 0 | \ No newline at end of file |
4667 | 27 | 1 | ||
4668 | === modified file 'IPython/kernel/engineconnector.py' | |||
4669 | --- IPython/kernel/engineconnector.py 2009-04-18 21:53:37 +0000 | |||
4670 | +++ IPython/kernel/engineconnector.py 2009-11-21 00:11:11 +0000 | |||
4671 | @@ -1,32 +1,38 @@ | |||
4672 | 1 | #!/usr/bin/env python | ||
4673 | 1 | # encoding: utf-8 | 2 | # encoding: utf-8 |
4674 | 2 | 3 | ||
4675 | 3 | """A class that manages the engines connection to the controller.""" | 4 | """A class that manages the engines connection to the controller.""" |
4676 | 4 | 5 | ||
4681 | 5 | __docformat__ = "restructuredtext en" | 6 | #----------------------------------------------------------------------------- |
4682 | 6 | 7 | # Copyright (C) 2008-2009 The IPython Development Team | |
4679 | 7 | #------------------------------------------------------------------------------- | ||
4680 | 8 | # Copyright (C) 2008 The IPython Development Team | ||
4683 | 9 | # | 8 | # |
4684 | 10 | # Distributed under the terms of the BSD License. The full license is in | 9 | # Distributed under the terms of the BSD License. The full license is in |
4685 | 11 | # the file COPYING, distributed as part of this software. | 10 | # the file COPYING, distributed as part of this software. |
4687 | 12 | #------------------------------------------------------------------------------- | 11 | #----------------------------------------------------------------------------- |
4688 | 13 | 12 | ||
4690 | 14 | #------------------------------------------------------------------------------- | 13 | #----------------------------------------------------------------------------- |
4691 | 15 | # Imports | 14 | # Imports |
4693 | 16 | #------------------------------------------------------------------------------- | 15 | #----------------------------------------------------------------------------- |
4694 | 17 | 16 | ||
4695 | 18 | import os | 17 | import os |
4696 | 19 | import cPickle as pickle | 18 | import cPickle as pickle |
4697 | 20 | 19 | ||
4698 | 21 | from twisted.python import log, failure | 20 | from twisted.python import log, failure |
4699 | 22 | from twisted.internet import defer | 21 | from twisted.internet import defer |
4700 | 22 | from twisted.internet.defer import inlineCallbacks, returnValue | ||
4701 | 23 | 23 | ||
4703 | 24 | from IPython.kernel.fcutil import find_furl | 24 | from IPython.kernel.fcutil import find_furl, validate_furl_or_file |
4704 | 25 | from IPython.kernel.enginefc import IFCEngine | 25 | from IPython.kernel.enginefc import IFCEngine |
4705 | 26 | from IPython.kernel.twistedutil import sleep_deferred, make_deferred | ||
4706 | 26 | 27 | ||
4708 | 27 | #------------------------------------------------------------------------------- | 28 | #----------------------------------------------------------------------------- |
4709 | 28 | # The ClientConnector class | 29 | # The ClientConnector class |
4711 | 29 | #------------------------------------------------------------------------------- | 30 | #----------------------------------------------------------------------------- |
4712 | 31 | |||
4713 | 32 | |||
4714 | 33 | class EngineConnectorError(Exception): | ||
4715 | 34 | pass | ||
4716 | 35 | |||
4717 | 30 | 36 | ||
4718 | 31 | class EngineConnector(object): | 37 | class EngineConnector(object): |
4719 | 32 | """Manage an engines connection to a controller. | 38 | """Manage an engines connection to a controller. |
4720 | @@ -38,8 +44,10 @@ | |||
4721 | 38 | 44 | ||
4722 | 39 | def __init__(self, tub): | 45 | def __init__(self, tub): |
4723 | 40 | self.tub = tub | 46 | self.tub = tub |
4726 | 41 | 47 | ||
4727 | 42 | def connect_to_controller(self, engine_service, furl_or_file): | 48 | @make_deferred |
4728 | 49 | def connect_to_controller(self, engine_service, furl_or_file, | ||
4729 | 50 | delay=0.1, max_tries=10): | ||
4730 | 43 | """ | 51 | """ |
4731 | 44 | Make a connection to a controller specified by a furl. | 52 | Make a connection to a controller specified by a furl. |
4732 | 45 | 53 | ||
4733 | @@ -48,34 +56,73 @@ | |||
4734 | 48 | foolscap URL contains all the information needed to connect to the | 56 | foolscap URL contains all the information needed to connect to the |
4735 | 49 | controller, including the ip and port as well as any encryption and | 57 | controller, including the ip and port as well as any encryption and |
4736 | 50 | authentication information needed for the connection. | 58 | authentication information needed for the connection. |
4738 | 51 | 59 | ||
4739 | 52 | After getting a reference to the controller, this method calls the | 60 | After getting a reference to the controller, this method calls the |
4740 | 53 | `register_engine` method of the controller to actually register the | 61 | `register_engine` method of the controller to actually register the |
4741 | 54 | engine. | 62 | engine. |
4748 | 55 | 63 | ||
4749 | 56 | :Parameters: | 64 | This method will try to connect to the controller multiple times with |
4750 | 57 | engine_service : IEngineBase | 65 | a delay in between. Each time the FURL file is read anew. |
4751 | 58 | An instance of an `IEngineBase` implementer | 66 | |
4752 | 59 | furl_or_file : str | 67 | Parameters |
4753 | 60 | A furl or a filename containing a furl | 68 | __________ |
4754 | 69 | engine_service : IEngineBase | ||
4755 | 70 | An instance of an `IEngineBase` implementer | ||
4756 | 71 | furl_or_file : str | ||
4757 | 72 | A furl or a filename containing a furl | ||
4758 | 73 | delay : float | ||
4759 | 74 | The intial time to wait between connection attempts. Subsequent | ||
4760 | 75 | attempts have increasing delays. | ||
4761 | 76 | max_tries : int | ||
4762 | 77 | The maximum number of connection attempts. | ||
4763 | 78 | |||
4764 | 79 | Returns | ||
4765 | 80 | ------- | ||
4766 | 81 | A deferred to the registered client or a failure to an error | ||
4767 | 82 | like :exc:`FURLError`. | ||
4768 | 61 | """ | 83 | """ |
4769 | 62 | if not self.tub.running: | 84 | if not self.tub.running: |
4770 | 63 | self.tub.startService() | 85 | self.tub.startService() |
4771 | 64 | self.engine_service = engine_service | 86 | self.engine_service = engine_service |
4772 | 65 | self.engine_reference = IFCEngine(self.engine_service) | 87 | self.engine_reference = IFCEngine(self.engine_service) |
4777 | 66 | try: | 88 | |
4778 | 67 | self.furl = find_furl(furl_or_file) | 89 | validate_furl_or_file(furl_or_file) |
4779 | 68 | except ValueError: | 90 | d = self._try_to_connect(furl_or_file, delay, max_tries, attempt=0) |
4780 | 69 | return defer.fail(failure.Failure()) | 91 | d.addCallback(self._register) |
4781 | 92 | return d | ||
4782 | 93 | |||
4783 | 94 | @inlineCallbacks | ||
4784 | 95 | def _try_to_connect(self, furl_or_file, delay, max_tries, attempt): | ||
4785 | 96 | """Try to connect to the controller with retry logic.""" | ||
4786 | 97 | if attempt < max_tries: | ||
4787 | 98 | log.msg("Attempting to connect to controller [%r]: %s" % \ | ||
4788 | 99 | (attempt, furl_or_file)) | ||
4789 | 100 | try: | ||
4790 | 101 | self.furl = find_furl(furl_or_file) | ||
4791 | 102 | # Uncomment this to see the FURL being tried. | ||
4792 | 103 | # log.msg("FURL: %s" % self.furl) | ||
4793 | 104 | rr = yield self.tub.getReference(self.furl) | ||
4794 | 105 | except: | ||
4795 | 106 | if attempt==max_tries-1: | ||
4796 | 107 | # This will propagate the exception all the way to the top | ||
4797 | 108 | # where it can be handled. | ||
4798 | 109 | raise | ||
4799 | 110 | else: | ||
4800 | 111 | yield sleep_deferred(delay) | ||
4801 | 112 | rr = yield self._try_to_connect( | ||
4802 | 113 | furl_or_file, 1.5*delay, max_tries, attempt+1 | ||
4803 | 114 | ) | ||
4804 | 115 | # rr becomes an int when there is a connection!!! | ||
4805 | 116 | returnValue(rr) | ||
4806 | 117 | else: | ||
4807 | 118 | returnValue(rr) | ||
4808 | 70 | else: | 119 | else: |
4817 | 71 | d = self.tub.getReference(self.furl) | 120 | raise EngineConnectorError( |
4818 | 72 | d.addCallbacks(self._register, self._log_failure) | 121 | 'Could not connect to controller, max_tries (%r) exceeded. ' |
4819 | 73 | return d | 122 | 'This usually means that i) the controller was not started, ' |
4820 | 74 | 123 | 'or ii) a firewall was blocking the engine from connecting ' | |
4821 | 75 | def _log_failure(self, reason): | 124 | 'to the controller.' % max_tries |
4822 | 76 | log.err('EngineConnector: engine registration failed:') | 125 | ) |
4815 | 77 | log.err(reason) | ||
4816 | 78 | return reason | ||
4823 | 79 | 126 | ||
4824 | 80 | def _register(self, rr): | 127 | def _register(self, rr): |
4825 | 81 | self.remote_ref = rr | 128 | self.remote_ref = rr |
4826 | @@ -83,7 +130,7 @@ | |||
4827 | 83 | desired_id = self.engine_service.id | 130 | desired_id = self.engine_service.id |
4828 | 84 | d = self.remote_ref.callRemote('register_engine', self.engine_reference, | 131 | d = self.remote_ref.callRemote('register_engine', self.engine_reference, |
4829 | 85 | desired_id, os.getpid(), pickle.dumps(self.engine_service.properties,2)) | 132 | desired_id, os.getpid(), pickle.dumps(self.engine_service.properties,2)) |
4831 | 86 | return d.addCallbacks(self._reference_sent, self._log_failure) | 133 | return d.addCallback(self._reference_sent) |
4832 | 87 | 134 | ||
4833 | 88 | def _reference_sent(self, registration_dict): | 135 | def _reference_sent(self, registration_dict): |
4834 | 89 | self.engine_service.id = registration_dict['id'] | 136 | self.engine_service.id = registration_dict['id'] |
4835 | 90 | 137 | ||
4836 | === modified file 'IPython/kernel/error.py' | |||
4837 | --- IPython/kernel/error.py 2009-08-04 01:18:28 +0000 | |||
4838 | +++ IPython/kernel/error.py 2009-11-21 00:11:11 +0000 | |||
4839 | @@ -127,9 +127,11 @@ | |||
4840 | 127 | class CompositeError(KernelError): | 127 | class CompositeError(KernelError): |
4841 | 128 | def __init__(self, message, elist): | 128 | def __init__(self, message, elist): |
4842 | 129 | Exception.__init__(self, *(message, elist)) | 129 | Exception.__init__(self, *(message, elist)) |
4844 | 130 | self.message = message | 130 | # Don't use pack_exception because it will conflict with the .message |
4845 | 131 | # attribute that is being deprecated in 2.6 and beyond. | ||
4846 | 132 | self.msg = message | ||
4847 | 131 | self.elist = elist | 133 | self.elist = elist |
4849 | 132 | 134 | ||
4850 | 133 | def _get_engine_str(self, ev): | 135 | def _get_engine_str(self, ev): |
4851 | 134 | try: | 136 | try: |
4852 | 135 | ei = ev._ipython_engine_info | 137 | ei = ev._ipython_engine_info |
4853 | @@ -137,7 +139,7 @@ | |||
4854 | 137 | return '[Engine Exception]' | 139 | return '[Engine Exception]' |
4855 | 138 | else: | 140 | else: |
4856 | 139 | return '[%i:%s]: ' % (ei['engineid'], ei['method']) | 141 | return '[%i:%s]: ' % (ei['engineid'], ei['method']) |
4858 | 140 | 142 | ||
4859 | 141 | def _get_traceback(self, ev): | 143 | def _get_traceback(self, ev): |
4860 | 142 | try: | 144 | try: |
4861 | 143 | tb = ev._ipython_traceback_text | 145 | tb = ev._ipython_traceback_text |
4862 | @@ -145,14 +147,14 @@ | |||
4863 | 145 | return 'No traceback available' | 147 | return 'No traceback available' |
4864 | 146 | else: | 148 | else: |
4865 | 147 | return tb | 149 | return tb |
4867 | 148 | 150 | ||
4868 | 149 | def __str__(self): | 151 | def __str__(self): |
4870 | 150 | s = str(self.message) | 152 | s = str(self.msg) |
4871 | 151 | for et, ev, etb in self.elist: | 153 | for et, ev, etb in self.elist: |
4872 | 152 | engine_str = self._get_engine_str(ev) | 154 | engine_str = self._get_engine_str(ev) |
4873 | 153 | s = s + '\n' + engine_str + str(et.__name__) + ': ' + str(ev) | 155 | s = s + '\n' + engine_str + str(et.__name__) + ': ' + str(ev) |
4874 | 154 | return s | 156 | return s |
4876 | 155 | 157 | ||
4877 | 156 | def print_tracebacks(self, excid=None): | 158 | def print_tracebacks(self, excid=None): |
4878 | 157 | if excid is None: | 159 | if excid is None: |
4879 | 158 | for (et,ev,etb) in self.elist: | 160 | for (et,ev,etb) in self.elist: |
4880 | 159 | 161 | ||
4881 | === modified file 'IPython/kernel/fcutil.py' | |||
4882 | --- IPython/kernel/fcutil.py 2008-06-06 19:41:55 +0000 | |||
4883 | +++ IPython/kernel/fcutil.py 2009-11-21 00:11:11 +0000 | |||
4884 | @@ -1,27 +1,62 @@ | |||
4885 | 1 | #!/usr/bin/env python | ||
4886 | 1 | # encoding: utf-8 | 2 | # encoding: utf-8 |
4894 | 2 | 3 | """ | |
4895 | 3 | """Foolscap related utilities.""" | 4 | Foolscap related utilities. |
4896 | 4 | 5 | """ | |
4897 | 5 | __docformat__ = "restructuredtext en" | 6 | |
4898 | 6 | 7 | #----------------------------------------------------------------------------- | |
4899 | 7 | #------------------------------------------------------------------------------- | 8 | # Copyright (C) 2008-2009 The IPython Development Team |
4893 | 8 | # Copyright (C) 2008 The IPython Development Team | ||
4900 | 9 | # | 9 | # |
4901 | 10 | # Distributed under the terms of the BSD License. The full license is in | 10 | # Distributed under the terms of the BSD License. The full license is in |
4902 | 11 | # the file COPYING, distributed as part of this software. | 11 | # the file COPYING, distributed as part of this software. |
4904 | 12 | #------------------------------------------------------------------------------- | 12 | #----------------------------------------------------------------------------- |
4905 | 13 | 13 | ||
4907 | 14 | #------------------------------------------------------------------------------- | 14 | #----------------------------------------------------------------------------- |
4908 | 15 | # Imports | 15 | # Imports |
4910 | 16 | #------------------------------------------------------------------------------- | 16 | #----------------------------------------------------------------------------- |
4911 | 17 | |||
4912 | 18 | from __future__ import with_statement | ||
4913 | 17 | 19 | ||
4914 | 18 | import os | 20 | import os |
4915 | 21 | import tempfile | ||
4916 | 22 | |||
4917 | 23 | from twisted.internet import reactor, defer | ||
4918 | 24 | from twisted.python import log | ||
4919 | 19 | 25 | ||
4920 | 20 | from foolscap import Tub, UnauthenticatedTub | 26 | from foolscap import Tub, UnauthenticatedTub |
4921 | 21 | 27 | ||
4922 | 28 | from IPython.config.loader import Config | ||
4923 | 29 | |||
4924 | 30 | from IPython.kernel.configobjfactory import AdaptedConfiguredObjectFactory | ||
4925 | 31 | |||
4926 | 32 | from IPython.kernel.error import SecurityError | ||
4927 | 33 | |||
4928 | 34 | from IPython.utils.traitlets import Int, Str, Bool, Instance | ||
4929 | 35 | from IPython.utils.importstring import import_item | ||
4930 | 36 | |||
4931 | 37 | #----------------------------------------------------------------------------- | ||
4932 | 38 | # Code | ||
4933 | 39 | #----------------------------------------------------------------------------- | ||
4934 | 40 | |||
4935 | 41 | |||
4936 | 42 | # We do this so if a user doesn't have OpenSSL installed, it will try to use | ||
4937 | 43 | # an UnauthenticatedTub. But, they will still run into problems if they | ||
4938 | 44 | # try to use encrypted furls. | ||
4939 | 45 | try: | ||
4940 | 46 | import OpenSSL | ||
4941 | 47 | except: | ||
4942 | 48 | Tub = UnauthenticatedTub | ||
4943 | 49 | have_crypto = False | ||
4944 | 50 | else: | ||
4945 | 51 | have_crypto = True | ||
4946 | 52 | |||
4947 | 53 | |||
4948 | 54 | class FURLError(Exception): | ||
4949 | 55 | pass | ||
4950 | 56 | |||
4951 | 57 | |||
4952 | 22 | def check_furl_file_security(furl_file, secure): | 58 | def check_furl_file_security(furl_file, secure): |
4953 | 23 | """Remove the old furl_file if changing security modes.""" | 59 | """Remove the old furl_file if changing security modes.""" |
4954 | 24 | |||
4955 | 25 | if os.path.isfile(furl_file): | 60 | if os.path.isfile(furl_file): |
4956 | 26 | f = open(furl_file, 'r') | 61 | f = open(furl_file, 'r') |
4957 | 27 | oldfurl = f.read().strip() | 62 | oldfurl = f.read().strip() |
4958 | @@ -29,41 +64,210 @@ | |||
4959 | 29 | if (oldfurl.startswith('pb://') and not secure) or (oldfurl.startswith('pbu://') and secure): | 64 | if (oldfurl.startswith('pb://') and not secure) or (oldfurl.startswith('pbu://') and secure): |
4960 | 30 | os.remove(furl_file) | 65 | os.remove(furl_file) |
4961 | 31 | 66 | ||
4962 | 67 | |||
4963 | 32 | def is_secure(furl): | 68 | def is_secure(furl): |
4964 | 69 | """Is the given FURL secure or not.""" | ||
4965 | 33 | if is_valid(furl): | 70 | if is_valid(furl): |
4966 | 34 | if furl.startswith("pb://"): | 71 | if furl.startswith("pb://"): |
4967 | 35 | return True | 72 | return True |
4968 | 36 | elif furl.startswith("pbu://"): | 73 | elif furl.startswith("pbu://"): |
4969 | 37 | return False | 74 | return False |
4970 | 38 | else: | 75 | else: |
4972 | 39 | raise ValueError("invalid furl: %s" % furl) | 76 | raise FURLError("invalid FURL: %s" % furl) |
4973 | 77 | |||
4974 | 40 | 78 | ||
4975 | 41 | def is_valid(furl): | 79 | def is_valid(furl): |
4976 | 80 | """Is the str a valid FURL or not.""" | ||
4977 | 42 | if isinstance(furl, str): | 81 | if isinstance(furl, str): |
4978 | 43 | if furl.startswith("pb://") or furl.startswith("pbu://"): | 82 | if furl.startswith("pb://") or furl.startswith("pbu://"): |
4979 | 44 | return True | 83 | return True |
4980 | 45 | else: | 84 | else: |
4981 | 46 | return False | 85 | return False |
4982 | 47 | 86 | ||
4983 | 87 | |||
4984 | 48 | def find_furl(furl_or_file): | 88 | def find_furl(furl_or_file): |
4985 | 89 | """Find, validate and return a FURL in a string or file.""" | ||
4986 | 49 | if isinstance(furl_or_file, str): | 90 | if isinstance(furl_or_file, str): |
4987 | 50 | if is_valid(furl_or_file): | 91 | if is_valid(furl_or_file): |
4988 | 51 | return furl_or_file | 92 | return furl_or_file |
4989 | 52 | if os.path.isfile(furl_or_file): | 93 | if os.path.isfile(furl_or_file): |
4991 | 53 | furl = open(furl_or_file, 'r').read().strip() | 94 | with open(furl_or_file, 'r') as f: |
4992 | 95 | furl = f.read().strip() | ||
4993 | 54 | if is_valid(furl): | 96 | if is_valid(furl): |
4994 | 55 | return furl | 97 | return furl |
4995 | 56 | raise ValueError("not a furl or a file containing a furl: %s" % furl_or_file) | ||
4996 | 57 | |||
4997 | 58 | # We do this so if a user doesn't have OpenSSL installed, it will try to use | ||
4998 | 59 | # an UnauthenticatedTub. But, they will still run into problems if they | ||
4999 | 60 | # try to use encrypted furls. | ||
5000 | 61 | try: |
This is the result of a massive effort to refactor IPython.kernel to use the new config system. This also includes an improved ipcluster/ ipcontroller/ ipengine command line programs. Here are the main files that should be reviewed:
IPython.kernel:
* clusterdir.py
* ipcontrollerapp.py
* ipengineapp.py
* launcher.py
* ipclusterapp.py
IPython. config. default:
* ipcontroller_ config. py
* ipengine_config.py
* ipcluster_config.py
I have also fixed a number of bugs and problems in IPython itself:
* The command line options use a new style. %unload_ ext/%reload_ ext/%install_ profiles/ %install_ default_ config.
* %debug is fixed.
* New magics %load_ext/
Enjoy!