possible_topdir removed from nova-api

Please help me understand how nova-api is being run so that you are unable to perform the same tasks as prior to my intentional removal of this piece of code.

Also, "BAD!" as a bug description really doesn't help me understand what is going on here. I've worked with a number of python projects and none of them have needed this to work.

Thanks!

This bug was more or less just a quick placeholder to link the branch for fast merging. We had already figured out the solution by this point.

The problem was nova-api was unable to import nova.services when attempting to start outside of the development branch (preventing nova-api from running entirely). If you were to install the package using setup.py (or any other real package) this would have gone unnoticed, since the modules would be in the python path.

Hey Josh,

nova-api being unable to import nova.services is the expected outcome of standard Python. Can you explain (or help me understand, because really that's all I want) how "setup.py develop" doesn't apply here? I don't understand why we even have a setup.py if we're not going to use it and bypass it's functionality with hacking the sys.path.

For those curious, setup.py/distutils provides the "develop" command which allows for installation of python projects into local/user-specific/project-specific site-packages directories. For example, on most Linux distributions it's as easy as:

$ python setup.py develop --user

This will install Nova in to /home/<user>/.local/lib/python2.x/site-packages. From there on out you'll be able to run bin/nova-api with the sys.path modifications removed.

For those without the "--user" flag (OSX and maybe older version of Python?), the documented way to use setup.py for development is to create a distutils.cfg file as described here: http://docs.python.org/install/index.html#location-and-names-of-config-files, then run:

$ python setup.py develop

...in the desired branch.

If your distutils.cfg file looks like this:

[develop]
prefix=/home/<user>/.local

then you'll get the same results.

I can't be the only one that things modifications to sys.path are sub-optimal, but I do have the bias of being rooted in my ways :)

Personally I don't want to have to run setup.py develop every time I edit some code and want to run it. If we really want to remove this functionality it should be discussed and done for all services simultaneously, because there is the expectation on the part of many devs (and scripts like nova.sh) that the code can be run directly from a source checkout/tarball

Vish

On Jun 28, 2011, at 6:32 PM, Brian Lamar wrote:

> For those curious, setup.py/distutils provides the "develop" command
> which allows for installation of python projects into local/user-
> specific/project-specific site-packages directories. For example, on
> most Linux distributions it's as easy as:
>
> $ python setup.py develop --user
>
> This will install Nova in to /home/<user>/.local/lib/python2.x/site-
> packages. From there on out you'll be able to run bin/nova-api with the
> sys.path modifications removed.
>
> For those without the "--user" flag (OSX and maybe older version of
> Python?), the documented way to use setup.py for development is to
> create a distutils.cfg file as described here:
> http://docs.python.org/install/index.html#location-and-names-of-config-
> files, then run:
>
> $ python setup.py develop
>
> ...in the desired branch.
>
> If your distutils.cfg file looks like this:
>
> [develop]
> prefix=/home/<user>/.local
>
> then you'll get the same results.
>
> I can't be the only one that things modifications to sys.path are sub-
> optimal, but I do have the bias of being rooted in my ways :)
>
> --
> You received this bug notification because you are a member of Nova Bug
> Team, which is subscribed to OpenStack Compute (nova).
> https://bugs.launchpad.net/bugs/803186
>
> Title:
> possible_topdir removed from nova-api
>
> Status in OpenStack Compute (Nova):
> Fix Committed
>
> Bug description:
> BAD!
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/nova/+bug/803186/+subscriptions

Ok, sounds like I'm in the extreme minority here, but I'll just say that you don't have to run "setup.py develop" every time you edit code. You'll only need to run it when you switch branches.

Run setup.py develop and that will be your primary Nova python package until you run 'develop' elsewhere.

To have this code in production is dangerous and unecessary in my opinion. Why risk running a different version of Nova than the one installed in production?

$ export PYTHONPATH=.

This does the exact same thing without any setup.py magic or code editing.

I'm in the same boat as Vish when it comes to my dev env preferences. That's not to say using setup.py doesn't have its uses (in fact, I would like to give it a try). However removing that chunk of code caused a major regression (especially since it was only removed from API and not Compute, Network, Scheduler, etc). Personally I'm not opposed to having it removed, I just think it warrants a ML discussion to make sure we're covering all our bases.

On Wed, Jun 29, 2011 at 8:26 AM, Brian Lamar <email address hidden> wrote:
> Ok, sounds like I'm in the extreme minority here, but I'll just say that
> you don't have to run "setup.py develop" every time you edit code.
> You'll only need to run it when you switch branches.

You're not the only one. It annoys the crap out of me to have all the
"possible topdir" stuff all over the bins (and in Keystone, all over
various random modules in the codebase).

I know of not a single other Python library or project that does this.

> Run setup.py develop and that will be your primary Nova python package
> until you run 'develop' elsewhere.
>
> To have this code in production is dangerous and unecessary in my
> opinion. Why risk running a different version of Nova than the one
> installed in production?
>
> $ export PYTHONPATH=.
>
> This does the exact same thing without any setup.py magic or code
> editing.

++

-jay

Before this discussion is lost due to the bug being now closed, maybe we should create a new bug to track the opportunity of getting rid of possible_topdir stuff, or start a ML thread to see where the consensus lies ?

On Thu, Jun 30, 2011 at 7:06 AM, Thierry Carrez <email address hidden> wrote:
> Before this discussion is lost due to the bug being now closed, maybe we
> should create a new bug to track the opportunity of getting rid of
> possible_topdir stuff, or start a ML thread to see where the consensus
> lies ?

We can do that (ML discussion). It's pretty low priority, though...

Brian, you want to draft an email to the ML? I'll back you up?

-jay