Merge lp:~ipython-dev/ipython/module-reorg into lp:ipython/0.11

[Brian Granger]

[Brian Granger (ellisonbg)]

This branch has a IPEP (IPython Enhancement Proposal) that describes how the top-level modules in IPython will be reorganized into logical sub-packages. The actually changes have not been made, but we need to discuss these changes before work begins.

[Jörgen Stenarson (jorgen-stenarson)]

Looks good overall but I have the following comments

line 5 in reorg.txt is over 80 characters long.

I feel that the color handling modules are not really part of the core functionality they are nice but not essential so perhaps ColorANSI.py, PyColorize.py, excolors.py would be better placed in lib. I do not have strong feelings on this.

shadowns.py looks like a misspelling of shadows perhaps shadow_ns would be better.

The functions in platutils_win32.py and winconsole.py have a similar scope and could be put together.

/Jörgen

[Brian Granger (ellisonbg)]

Thanks for looking this over.

> line 5 in reorg.txt is over 80 characters long.

Good catch.

> I feel that the color handling modules are not really part of the core functionality they are nice but not essential so perhaps ColorANSI.py, PyColorize.py, excolors.py would be better placed in lib. I do not have strong feelings on this.

Yes, these could probably go both ways, or even in IPython.python/tools/utils.

> shadowns.py looks like a misspelling of shadows perhaps shadow_ns would be better.

Yes, but I really want to get away from the module names with
underscores if we can.

> The functions in platutils_win32.py and winconsole.py have a similar scope and could be put together.

Yes, except the idea with the platutils_*.py files is that you can
just import platutils and there should be a version for all platforms
that gets picked up from the appropriate underlying platutils_*.py
module. I don't think there is a linux/mac version of winconsole.py,
so I don't think this makes sense. But, I will definitely look at
this.

Brian

> /Jörgen
>
> --
> https://code.launchpad.net/~ellisonbg/ipython/module-reorg/+merge/5184
> You are subscribed to branch lp:ipython.
>

--
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
<email address hidden>
<email address hidden>

[Brian Granger (ellisonbg)]

> line 5 in reorg.txt is over 80 characters long.

Fixed now.

> I feel that the color handling modules are not really part of the core
> functionality they are nice but not essential so perhaps ColorANSI.py,
> PyColorize.py, excolors.py would be better placed in lib. I do not have strong
> feelings on this.

I added notes to indicate we should look at these carefully when the reorg happens to see where
they best fit.

> shadowns.py looks like a misspelling of shadows perhaps shadow_ns would be
> better.

Leaving as shadowns.py for pep 8's sake.

> The functions in platutils_win32.py and winconsole.py have a similar scope and
> could be put together.

Leaving for now.

> /Jörgen

[Fernando Perez (fdo.perez)]

> > line 5 in reorg.txt is over 80 characters long.
>
> Fixed now.

BTW, I've noticed this happens some times in your reST edits. It may be an issue with your editor config for text files. I have my editor line wrap at 79 for all files to remain consistent, because these extra-long lines cause all kinds of problems when diffing (since diff is line-oriented, it sees gigantic diffs for small changes inside the paragraphs).

I'm sorry that I haven't commented yet. I will, but likely not in another couple of days.

The reference counting and Twisted bugs turned out to be total nightmares and I've spent almost the last two days entirely on them. Considering that I have 3 grant deadlines coming up at work, it's getting a bit painful...

But thanks a lot for starting this discussion. I do have some ideas, and as soon as I can reach some solution on the other things I'll pitch back in.

[Brian Granger (ellisonbg)]

> The reference counting and Twisted bugs turned out to be total nightmares and I've spent almost the last two days entirely on them.  Considering that I have 3 grant deadlines coming up at work, it's getting a bit painful..

Wow, thanks for working on that. Do you want to talk about anything
related to the Twisted bug? I can imagine it could be very nasty.

> But thanks a lot for starting this discussion.  I do have some ideas, and as soon as I can reach some solution on the other things I'll pitch back in.

No problem, I have lots of other things to work on for now. Also, if
you don't get to it, it is not a big deal - we can handle the rest of
the feedback in code review :-)

Brian

> --
> https://code.launchpad.net/~ellisonbg/ipython/module-reorg/+merge/5184
> You are subscribed to branch lp:ipython.
>

--
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
<email address hidden>
<email address hidden>

[Fernando Perez (fdo.perez)]

Since we're having this discussion (for now) mostly on-list, perhaps we should just withdraw this one from 'merge request'. That way the active reviews page will show us the status of what's really on the table for 0.10.

Cheers,

f

[Fernando Perez (fdo.perez)]

Resubmit after 0.10, discussion is now on-list to get input from non-launchpad committers.

review: Needs Resubmitting

[Brian Granger (ellisonbg)]

I don't have a problem with that, but I have a question. On various
places on launchpad, I see references to "releases" and "series" and
"milestones." But I can't find anyway of manipulating these things to
create (for example) tickets/branches related to a particular
"release/series/milestone." Am I just missing something? Do I not
have enough LP privs? What do you think?

It would be nice to target tickets/branches for upcoming releases.

Cheers,

Brian

On Tue, Apr 14, 2009 at 5:53 PM, Fernando Perez <email address hidden> wrote:
> Since we're having this discussion (for now) mostly on-list, perhaps we should just withdraw this one from 'merge request'.  That way the active reviews page will show us the status of what's really on the table for 0.10.
>
> Cheers,
>
> f
> --
> https://code.launchpad.net/~ellisonbg/ipython/module-reorg/+merge/5184
> You are subscribed to branch lp:ipython.
>

--
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
<email address hidden>
<email address hidden>

[Fernando Perez (fdo.perez)]

On Tue, Apr 14, 2009 at 5:57 PM, Brian Granger <email address hidden> wrote:
> I don't have a problem with that, but I have a question.  On various
> places on launchpad, I see references to "releases" and "series" and
> "milestones."  But I can't find anyway of manipulating these things to
> create (for example) tickets/branches related to a particular
> "release/series/milestone."  Am I just missing something?  Do I not
> have enough LP privs?  What do you think?
>
> It would be nice to target tickets/branches for upcoming releases.

I've wondered exactly the same thing. It's one of the things I miss
from trac. It may be possible to do it in LP, I don't quite know
yet...

I doubt it's a matter of privs, you have the same I do (admin)...

We're having a nipy sprint, so my presence is kind of spotty... But
ping me if anything comes up, I'm trying to keep an eye on things.

f

[Brian Granger (ellisonbg)]

Here is the page describing releases and milestones and they sound great:

https://help.launchpad.net/Projects/SeriesMilestonesReleases

But I don't see anything on our pages that have these things. Are you
*sure* that I have the same privs. Could you look to see if the links
referred to in this doc are there for you. I know I am listed as an
Admin, just like you are, but this is really driving me crazy that I
can't find this.

Cheers,

Brian

On Tue, Apr 14, 2009 at 6:06 PM, Fernando Perez <email address hidden> wrote:
> On Tue, Apr 14, 2009 at 5:57 PM, Brian Granger <email address hidden> wrote:
>> I don't have a problem with that, but I have a question.  On various
>> places on launchpad, I see references to "releases" and "series" and
>> "milestones."  But I can't find anyway of manipulating these things to
>> create (for example) tickets/branches related to a particular
>> "release/series/milestone."  Am I just missing something?  Do I not
>> have enough LP privs?  What do you think?
>>
>> It would be nice to target tickets/branches for upcoming releases.
>
> I've wondered exactly the same thing.  It's one of the things I miss
> from trac.  It may be possible to do it in LP, I don't quite know
> yet...
>
> I doubt it's a matter of privs, you have the same I do (admin)...
>
> We're having a nipy sprint, so my presence is kind of spotty... But
> ping me if anything comes up, I'm trying to keep an eye on things.
>
> f
> --
> https://code.launchpad.net/~ellisonbg/ipython/module-reorg/+merge/5184
> You are subscribed to branch lp:ipython.
>

--
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
<email address hidden>
<email address hidden>

[Brian Granger (ellisonbg)]

I am chatting with the lp devs on freenode and it looks like only you
(as "owner") can create series. This seems stupid, but you should see
a link on the main page that says "Register series" Can you check?

Brian

On Tue, Apr 14, 2009 at 6:06 PM, Fernando Perez <email address hidden> wrote:
> On Tue, Apr 14, 2009 at 5:57 PM, Brian Granger <email address hidden> wrote:
>> I don't have a problem with that, but I have a question.  On various
>> places on launchpad, I see references to "releases" and "series" and
>> "milestones."  But I can't find anyway of manipulating these things to
>> create (for example) tickets/branches related to a particular
>> "release/series/milestone."  Am I just missing something?  Do I not
>> have enough LP privs?  What do you think?
>>
>> It would be nice to target tickets/branches for upcoming releases.
>
> I've wondered exactly the same thing.  It's one of the things I miss
> from trac.  It may be possible to do it in LP, I don't quite know
> yet...
>
> I doubt it's a matter of privs, you have the same I do (admin)...
>
> We're having a nipy sprint, so my presence is kind of spotty... But
> ping me if anything comes up, I'm trying to keep an eye on things.
>
> f
> --
> https://code.launchpad.net/~ellisonbg/ipython/module-reorg/+merge/5184
> You are subscribed to branch lp:ipython.
>

--
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
<email address hidden>
<email address hidden>

[Fernando Perez (fdo.perez)]

Hey,

On Tue, Apr 14, 2009 at 9:09 PM, Brian Granger <email address hidden> wrote:
> Here is the page describing releases and milestones and they sound great:
>
> https://help.launchpad.net/Projects/SeriesMilestonesReleases
>
> But I don't see anything on our pages that have these things.  Are you
> *sure* that I have the same privs.  Could you look to see if the links
> referred to in this doc are there for you.  I know I am listed as an
> Admin, just like you are, but this is really driving me crazy that I
> can't find this.

I do see a 'register series' button below the list of series, I just
made these two:

https://launchpad.net/ipython/0.9
https://launchpad.net/ipython/0.10

Once we actually branch 0.10 we can link a branch to it, but for now
it has no branch associated with it. I guess we should make the tag
and branch pretty soon. I'll send an email on-list and can take care
of it tomorrow...

Cheers,

f

[Fernando Perez (fdo.perez)]

On Tue, Apr 14, 2009 at 9:30 PM, Brian Granger <email address hidden> wrote:
> I am chatting with the lp devs on freenode and it looks like only you
> (as "owner") can create series.  This seems stupid, but you should see
> a link on the main page that says "Register series"  Can you check?

Ah, I did the work before seeing this email...

Yes, I just did the series creation, so we're good to go. Silly, but
making series should be a once in a while thing, so it's no big deal.
As long as you can then manage it normally (add milestones, edit
details, etc), I think we're good.

Cheers,

f

[Brian Granger (ellisonbg)]

This branch has a complete re-organization of all IPython's modules and
packages. It needs extensive review (people need to install it and run the
test suite and play around with it) and needs to be merged ASAP. Once this
is merged, other branches targeted to trunk will have to be re-done.

Cheers,

Brian

On Mon, Jul 27, 2009 at 3:56 PM, Brian Granger <email address hidden> wrote:

> The proposal to merge lp:~ipython-dev/ipython/module-reorg into lp:ipython
> has been updated.
>
> Status: Work in progress => Needs review
> --
> https://code.launchpad.net/~ipython-dev/ipython/module-reorg/+merge/5184<https://code.launchpad.net/%7Eipython-dev/ipython/module-reorg/+merge/5184>
> You proposed lp:~ipython-dev/ipython/module-reorg for merging.
>

--
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
<email address hidden>
<email address hidden>

[Vishal Vatsa (vvatsa)]

Looks good, I have had a quick poke around, ran some of my own code
ran the tests.

-vishal

2009/7/28 Brian Granger <email address hidden>:
> This branch has a complete re-organization of all IPython's modules and
> packages.  It needs extensive review (people need to install it and run the
> test suite and play around with it) and needs to be merged ASAP.  Once this
> is merged, other branches targeted to trunk will have to be re-done.
>
> Cheers,
>
> Brian
>
> On Mon, Jul 27, 2009 at 3:56 PM, Brian Granger <email address hidden> wrote:
>
>> The proposal to merge lp:~ipython-dev/ipython/module-reorg into lp:ipython
>> has been updated.
>>
>>    Status: Work in progress => Needs review
>> --
>> https://code.launchpad.net/~ipython-dev/ipython/module-reorg/+merge/5184<https://code.launchpad.net/%7Eipython-dev/ipython/module-reorg/+merge/5184>
>> You proposed lp:~ipython-dev/ipython/module-reorg for merging.
>>
>
>
>
> --
> Brian E. Granger, Ph.D.
> Assistant Professor of Physics
> Cal Poly State University, San Luis Obispo
> <email address hidden>
> <email address hidden>
>
> https://code.launchpad.net/~ipython-dev/ipython/module-reorg/+merge/5184
> You are subscribed to branch lp:ipython.
>

[Brian Granger (ellisonbg)]

Thanks!

On Tue, Jul 28, 2009 at 3:48 AM, Vishal Vatsa <email address hidden>wrote:

> Looks good, I have had a quick poke around, ran some of my own code
> ran the tests.
>
> -vishal
>
> 2009/7/28 Brian Granger <email address hidden>:
> > This branch has a complete re-organization of all IPython's modules and
> > packages. It needs extensive review (people need to install it and run
> the
> > test suite and play around with it) and needs to be merged ASAP. Once
> this
> > is merged, other branches targeted to trunk will have to be re-done.
> >
> > Cheers,
> >
> > Brian
> >
> > On Mon, Jul 27, 2009 at 3:56 PM, Brian Granger <email address hidden>
> wrote:
> >
> >> The proposal to merge lp:~ipython-dev/ipython/module-reorg into
> lp:ipython
> >> has been updated.
> >>
> >> Status: Work in progress => Needs review
> >> --
> >>
> https://code.launchpad.net/~ipython-dev/ipython/module-reorg/+merge/5184<https://code.launchpad.net/%7Eipython-dev/ipython/module-reorg/+merge/5184>
> <
> https://code.launchpad.net/%7Eipython-dev/ipython/module-reorg/+merge/5184
> >
> >> You proposed lp:~ipython-dev/ipython/module-reorg for merging.
> >>
> >
> >
> >
> > --
> > Brian E. Granger, Ph.D.
> > Assistant Professor of Physics
> > Cal Poly State University, San Luis Obispo
> > <email address hidden>
> > <email address hidden>
> >
> > https://code.launchpad.net/~ipython-dev/ipython/module-reorg/+merge/5184<https://code.launchpad.net/%7Eipython-dev/ipython/module-reorg/+merge/5184>
> > You are subscribed to branch lp:ipython.
> >
> --
> https://code.launchpad.net/~ipython-dev/ipython/module-reorg/+merge/5184<https://code.launchpad.net/%7Eipython-dev/ipython/module-reorg/+merge/5184>
> You proposed lp:~ipython-dev/ipython/module-reorg for merging.
>

--
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
<email address hidden>
<email address hidden>

[lp:~ipython-dev/ipython/module-reorg]

1223. By Brian Granger on 2009-08-03

Merging revision 1175 from lp:ipython.

1224. By Brian Granger on 2009-08-03

Merging revision 1176 from lp:ipython.

1225. By Brian Granger on 2009-08-03

Merging -r 1177 from lp:ipython with fixes and resolutions.

The main conflicts I had to fix were in ultratb. I have removed
the ultraTB.py in IPython/kernel/core. Now IPython/core/ultratb.py
is being used everywhere. Also I have protected the calls to ipapi.get
to see if None is returned. This happens when trial IPython.kernel
is run.

1226. By Brian Granger on 2009-08-03

Merging -r 1178 from lp:ipython.

No major conflicts.

1227. By Brian Granger on 2009-08-03

Merging -r 1179 from lp:ipython.

No conflicts.

1228. By Brian Granger on 2009-08-03

Merging -r 1180 from lp:ipthon

No conflicts!

1229. By Brian Granger on 2009-08-03

Merging -r 1181 from lp:ipython.

A few changes were made to resolve conflicts:

* clipboard.py moved to core.
* Imports fixed.

1230. By Brian Granger on 2009-08-03

Fixing small bug in import statement in clipboard.py.

1231. By Brian Granger on 2009-08-03

Merging -r 1182 from lp:ipython.

1232. By Brian Granger on 2009-08-03

Merging -r 1182 from lp:ipython.

Minor conflicts with removal of testing/attic.

1233. By Brian Granger on 2009-08-03

Merging -r 1184 from lp:ipython.

1234. By Brian Granger on 2009-08-04

Merging -r 1185 from lp:ipython.

1235. By Brian Granger on 2009-08-04

Merging -r 1186 from lp:ipython.

1236. By Brian Granger on 2009-08-04

Merging -r 1187 from lp:ipython.

1237. By Brian Granger on 2009-08-04

Merging -r 1188 from lp:ipython.

1238. By Brian Granger on 2009-08-04

Merging -r 1189 from lp:ipython.

1239. By Brian Granger on 2009-08-04

Merging -r 1190 from lp:ipython.

1240. By Brian Granger on 2009-08-04

Merging -r 1191 from lp:ipython.

1241. By Brian Granger on 2009-08-04

Merging -r 1192 from lp:ipython.

1242. By Brian Granger on 2009-08-04

Merging -r 1193 from lp:ipython.

1243. By Brian Granger on 2009-08-04

Merging -r 1194 from lp:ipython.

1244. By Brian Granger on 2009-08-04

Merging -r 1195 from lp:ipython.x

1245. By Brian Granger on 2009-08-04

Merging -r 1196 from lp:ipython.

A couple of issues came up:

* Some tests in testing and frontend rely on twisted, but are being
  tested with nose. This is bad! We currently have hackish logic in
  iptest to skip these if twisted is not installed, but if it is we
  are testing them with nose!
* Some modules (engineservice, kernel/error, newserialized) have nose
  skip logic even though they should never be tested with nose.
* When trial is run on testStrictDict we get an uncaught error.

testStrictDict ... ERROR: An unexpected error occurred while tokenizing
input The following traceback may be corrupted or invalid
The error message is: ('EOF in multi-line statement', (37, 0))

1246. By Brian Granger on 2009-08-04

Merging -r 1197 from lp:ipython.

1247. By Brian Granger on 2009-08-04

Merging -r 1198 from lp:ipython.

1248. By Brian Granger on 2009-08-04

Merging -r 1199 from lp:ipython.

1249. By Brian Granger on 2009-08-04

Merging -r 1200 from lp:ipython.

1250. By Brian Granger on 2009-08-04

Merging -r 1201 from lp:ipython.

This concludes the upstream merge of trunk into the module-reorg branch.

[Brian Granger (ellisonbg)]

I have merged all the recent activity in trunk into the module-reorg branch. The merge capabilities of bzr are quite impressive!!! Other than a few minor things, module-reorg is thus ready to be merged into trunk. I would like to merge this in the next day or so, so that development can begin again post-0.10. There are *lots* of commits in this branch and many of them are non-trivial. Please have a look if you get a chance.

[lp:~ipython-dev/ipython/module-reorg]

1251. By Brian Granger on 2009-08-05

Merging upstream changes from final 0.10 release.

[Fernando Perez (fdo.perez)]

First, some small comments:

- There's still a scripts/ package (empty) being installed. I think the plan
  was to move towards scripts being in the subpackage they belong, which I'm on
  board with. Do we need this 'scripts' package at all?

- .hgignore file can be nuked, since we're just using bzr and will be for a
  while. I don't know what the .checkeol file is; it seems to be a bzr thing
  but I'm not 100% sure.

- Stray quote in the sources/development/reorg.txt file, in the title (third
  line).

- This is a comment probably more for the config branch, but I'll jot it here
  so I don't forget. At some point we'll need to write up a little transition
  guide, and the user config files will be the place where people will get the
  most warnings and weirdness. I just went through cleaning my ~/.ipython
  directory and now it's fine, but users may get a little confused navigating
  that. Not a problem, just a reminder for us to do that once our own
  construction dust settles.

- The version number info should be updated on Release.py so users see it's a
  dev branch (can just pull from trunk that one file and update the bzr rev
  number with update_revno.py in the tools/ directory).

Other than the above minor nits, I'm voting to approve this merge (meaning, my
vote is approve, I trust you can take care of these tiny things in merging).

I think you've done a *stellar* job here, and that holding this merge further
won't help us any. There will be more work to be done, but that will be
easier done in-place, as we work through the consequences of these changes.

So +1 from me, and a huge thank you for doing such a thorough and careful job
with a task that was critically needed for the project, but both tricky due to
little details and simultaneously not very interesting.

Fantastic work!!

review: Approve

[lp:~ipython-dev/ipython/module-reorg]

1252. By Brian Granger on 2009-08-13

Removing .hgignore.

1253. By Brian Granger on 2009-08-13

Added to changes.txt.

[Brian Granger (ellisonbg)]

This one is in!

review: Approve

[Fernando Perez (fdo.perez)]

On Thu, Aug 13, 2009 at 4:23 PM, Brian Granger<email address hidden> wrote:
> Review: Approve
> This one is in!

Awesome! Great job.

=== modified file 'docs/source/development/index.txt'
--- docs/source/development/index.txt	2009-03-15 01:38:00 +0000
+++ docs/source/development/index.txt	2009-03-23 16:04:23 +0000
@@ -12,3 +12,4 @@
 
    notification_blueprint.txt
    config_blueprint.txt
+   reorg.txt

=== added file 'docs/source/development/reorg.txt'
--- docs/source/development/reorg.txt	1970-01-01 00:00:00 +0000
+++ docs/source/development/reorg.txt	2009-04-03 05:21:49 +0000
@@ -0,0 +1,218 @@
+=============================
+IPython module reorganization
+=============================
+
+Currently, IPython has many top-level modules that serve many different purposes.  The lack of organization make it very difficult for developers to work on IPython and understand its design.  This document contains notes about how we will reorganize the modules into sub-packages.
+
+.. warning::
+
+    This effort will possibly break third party packages that use IPython as
+    a library or hack on the IPython internals.
+
+.. warning::
+
+    This effort will result in the removal from IPython of certain modules
+    that are not used anymore, don't currently work, are unmaintained, etc.
+
+
+Current subpackges
+==================
+
+IPython currently has the following sub-packages:
+
+* :mod:`IPython.config`
+
+* :mod:`IPython.Extensions`
+
+* :mod:`IPython.external`
+
+* :mod:`IPython.frontend`
+
+* :mod:`IPython.gui`
+
+* :mod:`IPython.kernel`
+
+* :mod:`IPython.testing`
+
+* :mod:`IPython.tests`
+
+* :mod:`IPython.tools`
+
+* :mod:`IPython.UserConfig`
+
+New Subpackages to be created
+=============================
+
+We propose to create the following new sub-packages:
+
+* :mod:`IPython.core`. This sub-package will contain the core of the IPython
+  interpreter, but none of its extended capabilities.
+
+* :mod:`IPython.lib`. IPython has many extended capabilities that are not part
+  of the IPython core. These things will go here.  Any better names than 
+  :mod:`IPython.lib`?
+
+* :mod:`IPython.python`. This sub-package will contain anything that might
+  eventually be found in the Python standard library, like things in
+  :mod:`genutils`. Each sub-module in this sub-package should contain functions 
+  and classes that serve a single purpose. Similar in purpose to
+  :mod:`twisted.python`. Could also call this :mod:`IPython.tools`,
+  :mod:`IPython.utils` or something similar.
+
+* :mod:`IPython.sandbox`. This is for code that is untested and/or rotting and
+  needs to be removed from IPython. Eventually all this code will either i) be
+  revived with tests and docs and re-included into IPython or 2) be removed from
+  IPython proper, but put into a separate top-level (not IPython) package that we
+  keep around.
+
+Where things will be moved
+==========================
+
+* :file:`ColorANSI.py`.  Move to :file:`IPython/core/coloransi.py`.
+
+* :file:`ConfigLoader.py`.  Move to :file:`IPython/config/configloader.py`.
+
+* :file:`CrashHandler.py`.  Move to :file:`IPython/core/crashhandler`.
+
+* :file:`DPyGetOpt.py`.  Move to :mod:`IPython.sandbox` and replace with newer options parser.
+
+* :file:`Debugger.py`.  Move to :file:`IPython/core/debugger.py`.
+
+* :file:`Extensions`. This needs to be gone through separately.  Minimally,
+  the package should be renamed to :file:`extensions`.
+
+* :file:`FakeModule.py`.  Move to :file:`IPython/core/fakemodule.py`.
+
+* :file:`Gnuplot2.py`.  Move to :file:`IPython.sandbox`.
+
+* :file:`GnuplotInteractive.py`.  Move to :file:`IPython.sandbox`.
+
+* :file:`GnuplotRuntime.py`.  Move to :file:`IPython.sandbox`.
+
+* :file:`Itpl.py`.  Remove.  Version already in :file:`IPython.external`.
+
+* :file:`Logger.py`.  Move to :file:`IPython/core/logger.py`.
+
+* :file:`Magic.py`.  Move to :file:`IPython/core/magic.py`.
+
+* :file:`OInspect.py`.  Move to :file:`IPython/core/oinspect.py`.
+
+* :file:`OutputTrap.py`.  Move to :file:`IPython/core/outputtrap.py`.
+
+* :file:`Prompts.py`.  Move to :file:`IPython/core/prompts.py` or
+  :file:`IPython/frontend/prompts.py`.
+
+* :file:`PyColorize.py`.  Replace with pygments?  If not, move to :file:`IPython/core/pycolorize.py`.
+
+* :file:`Release.py`.  Move to ??? or remove?
+
+* :file:`Shell.py`.  Move to :file:`IPython.core.shell.py` or
+  :file:`IPython/frontend/shell.py`.
+
+* :file:`UserConfig`.  Move to a subdirectory of :file:`IPython.config`.
+
+* :file:`background_jobs.py`.  Move to :file:`IPython/lib/backgroundjobs.py`.
+
+* :file:`completer.py`.  Move to :file:`IPython/core/completer.py`.
+
+* :file:`config`.  Good where it is!
+
+* :file:`deep_reload.py`.  Move to :file:`IPython/lib/deepreload.py`.
+
+* :file:`demo.py`.  Move to :file:`IPython/lib/demo.py`.
+
+* :file:`dtutils.py`.  Remove or move to :file:`IPython.testing` or
+  :file:`IPython.lib`.
+
+* :file:`excolors.py`.  Move to :file:`IPython.core` or :file:`IPython.config`.
+
+* :file:`external`.  Good where it is!
+
+* :file:`frontend`.  Good where it is!
+
+* :file:`generics.py`.  Move to :file:`IPython.python`.
+
+* :file:`genutils.py`.  Move to :file:`IPython.python` and break up into different
+  modules.
+
+* :file:`gui`.  Eventually this should be moved to a subdir of   
+  :file:`IPython.frontend`.
+
+* :file:`history.py`.  Move to :file:`IPython.core`.
+
+* :file:`hooks.py`.  Move to :file:`IPython.core`.
+
+* :file:`ipapi.py`.  Move to :file:`IPython.core`.
+
+* :file:`iplib.py`.  Move to :file:`IPython.core`.
+
+* :file:`ipmaker.py`:  Move to :file:`IPython.core`.
+
+* :file:`ipstruct.py`.  Move to :file:`IPython.python`.
+
+* :file:`irunner.py`.  Move to :file:`IPython.scripts`.
+
+* :file:`kernel`.  Good where it is.
+
+* :file:`macro.py`.  Move to :file:`IPython.core`.
+
+* :file:`numutils.py`.  Move to :file:`IPython.sandbox`.
+
+* :file:`platutils.py`.  Move to :file:`IPython.python`.
+
+* :file:`platutils_dummy.py`.  Move to :file:`IPython.python`.
+
+* :file:`platutils_posix.py`.  Move to :file:`IPython.python`.
+
+* :file:`platutils_win32.py`.  Move to :file:`IPython.python`.
+
+* :file:`prefilter.py`:  Move to :file:`IPython.core`.
+
+* :file:`rlineimpl.py`.  Move to :file:`IPython.core`.
+
+* :file:`shadowns.py`.  Move to :file:`IPython.core`.
+
+* :file:`shellglobals.py`.  Move to :file:`IPython.core`.
+
+* :file:`strdispatch.py`.  Move to :file:`IPython.python`.
+
+* :file:`testing`.  Good where it is.
+
+* :file:`tests`.  Good where it is.
+
+* :file:`tools`.  Things in here need to be looked at and moved elsewhere like
+  :file:`IPython.python`.
+
+* :file:`twshell.py`.  Move to :file:`IPython.sandbox`.
+
+* :file:`ultraTB.py`.  Move to :file:`IPython/core/ultratb.py`.
+
+* :file:`upgrade_dir.py`.  Move to :file:`IPython/python/upgradedir.py`.
+
+* :file:`usage.py`.  Move to :file:`IPython.core`.
+
+* :file:`wildcard.py`.  Move to :file:`IPython.python` or :file:`IPython.core`.
+
+* :file:`winconsole.py`.  Move to :file:`IPython.lib`.
+
+Other things
+============
+
+When these files are moved around, a number of other things will happen at the same time:
+
+1. Test files will be created for each module in IPython. Minimally, all
+   modules will be imported as a part of the test. This will serve as a 
+   test of the module reorganization. These tests will be put into new
+   :file:`tests` subdirectories that each package will have.
+
+2. PyFlakes and other code checkers will be run to look for problems.
+
+3. Modules will be renamed to comply with PEP 8 naming conventions:  all
+   lowercase and no special characters like ``-`` or ``_``.
+
+4. Existing tests will be moved to the appropriate :file:`tests`
+   subdirectories.
+
+
+
+