temp file stored in python script directory?

Hi,

you may customize this.

However, think there is a bug though.

While it makes some reasonable proposals of default value at linux systems, offers the current directory
at MS.

WDYT is a suitable default for you?

>you may customize this.
>
>However, think there is a bug though.
>
>While it makes some reasonable proposals of default value at linux
>systems, offers the current directory
>at MS.
>
>WDYT is a suitable default for you?

I found the custom entry for Py Temp Directory. Under Windows, it offers
me a default of c:/Users/stanton/AppData/Local/Temp, and under OS X it
offers a default of /var/folders/zf/bgjm4tvs3wv_6q7_6z3b2nx00000gn/T/.
Both of those sound reasonable, as they're the current settings of
TEMP/TMP and TMPDIR on the two machines respectively. However, this
doesn't seem to be where the files are actually being stored.

There seem to be two issues here:

1) The directory is not being applied correctly.

2) The file name contains directory separators (which are also in the
buffer name) so if the directory *was* being used properly, Emacs would
throw an error here. In creating the name of the temp file from the buffer
name, it looks like "*" characters have been removed, which is good, but I
think "/" characters should also be removed, or else converted to
something more innocuous like "_", as otherwise Emacs thinks it's saving a
file in a multiply nested directory that doesn't exist.
>

Am 19.03.2012 17:31, schrieb Richard Stanton:
[ ... ]
  In creating the name of the temp file from the buffer
> name, it looks like "*" characters have been removed, which is good, but I
> think "/" characters should also be removed,

that should not occur. If it does, please make a new report, send some recipe to reproduce,

thanks

>
> 1) The directory is not being applied correctly.
>

need a recipe for it, a new report would help, as it's related only

thanks

> > 1) The directory is not being applied correctly.
> >
>
> need a recipe for it, a new report would help, as it's related only

Actually, I think this is the original problem I was reporting. Any python script will do in my experience, but the problem only occurs if the script file does *not* contain a shebang line. For example, the following file works fine:

--------

#!/usr/bin/python
print "Hi, Richard"

---------

In this case, the temp file is created in the appropriate temp directory.

However, if I use the same file without the shebang line,

--------

print "Hi, Richard"

---------

now the problem occurs.

I've been tracing through the code to see where things are going wrong, and have found some interesting things inside the function py-execute-base, which seems to be where all this is happening.

First, is there a bug in the function py-separator-char? On my Mac, this function seems to return

"Enthought Python Distribution -- www.enthought.com/",

which is then the value given to the variable sepchar. This doesn't look like any separator character I've ever seen!

The I tried tracing through py-execute-base with two different input files, one with NO shebang, and one with a shebang line:

1) No shebang:

Line 7181: (name (py-buffer-name-prepare shell sepchar)
shell = "/Library/Frameworks/EPD64.framework/Versions/7.2/bin/ipython"
sepchar = "Enthought Python Distribution -- www.enthought.com/"

Returns "*/Library/Frameworks/EPD64.framework/Versions/7.2/bin/ipython*"

(temp (Make-temp-name shell)) sets temp to "/Library/Frameworks/EPD64.framework/Versions/7.2/bin/ipython"

temp = "/Library/Frameworks/EPD64.framework/Versions/7.2/bin/ipython60832dhn"
py-temp-directory is "/var/folders/zf/bgjm4tvs3wv_6q7_6z3b2nx00000gn/T/"

(expand-file-name temp py-temp-directory) returns "/Library/Frameworks/EPD64.framework/Versions/7.2/bin/ipython60832dhn"

2) *With* shebang:

shell = "ipython"

(name (py-buffer-name-prepare shell sepchar)
shell = "/Library/Frameworks/EPD64.framework/Versions/7.2/bin/ipython"
sepchar = "Enthought Python Distribution -- www.enthought.com/"

Returns "*Ipython*"

(temp (Make-temp-name shell)) sets temp to "ipython60832qrt"

(expand-file-name temp py-temp-directory) returns "var/folders/zf/bgjm4tvs3wv_6q7_6z3b2nx00000gn/T/ipython60832qrt.py"

So the problems with setting the temp file name/path are definitely in this section of code somewhere.

I think all the problems are caused by the fact that shell has "/" characters in it. If these were replaced with, say, "_" before being passed to make-temp-name and expand-file-name, I suspect everything would then work OK.

On my Windows machine, inside py-execute-base, I tried replaceding

       (temp (make-temp-name shell))

with

          (replace-regexp-in-string ":" "!"
                                    (replace-regexp-in-string
                                     "/" "!" (make-temp-name shell))))

and things work much better. This code replaces the ":" and "/" characters in the shell name with the "!" character, which is legal inside a file name.

Am 21.03.2012 19:04, schrieb Richard Stanton:
> I think all the problems are caused by the fact that shell has "/"
> characters in it.

who did them into? :)

What about just customizing py-shell-name according to the environment?

The problem is still there in certain situations, I'm afraid, with r918 on my Mac. Here are three experiments. In numbers 1 and 2, things work as they're supposed to. In number 3, they don't:

1) Script file =

#!/usr/bin/ipython
print "Hi, Richard"

In this case, the temp file is correctly stored in the temp directory as (for example)
/var/folders/zf/bgjm4tvs3wv_6q7_6z3b2nx00000gn/T/ipython61090CmR.py. The ipython buffer is called *IPython*.

2) Script file =

print "Hi, Richard"

py-shell-name = "ipython"

In this case, the temp file is again correctly stored in the temp directory as (for example)
/var/folders/zf/bgjm4tvs3wv_6q7_6z3b2nx00000gn/T/ipython61090CmR.py. The ipython buffer is again named *IPython*

3) Script file =

print "Hi, Richard"

py-shell-name = "/Library/Frameworks/EPD64.framework/Versions/7.2/bin/ipython"

In this case, the temp file is saved in the *wrong* location as
/Library/Frameworks/EPD64.framework/Versions/7.2/bin/ipython61090PwX.py.

In this case, the ipython buffer is called *ND 0.12*

will split the temp-file-issue into two parts

a customizable directory will precede current guessing

r920: I set py-custom-temp-directory, and the file was still saved in the python script directory

Am 06.04.2012 19:41, schrieb Richard Stanton:
> r920: I set py-custom-temp-directory, and the file was still saved in
> the python script directory
>
> ** Changed in: python-mode
> Status: Fix Committed => In Progress
>

did you re-compile before loading?

>Am 06.04.2012 19:41, schrieb Richard Stanton:
>> r920: I set py-custom-temp-directory, and the file was still saved in
>> the python script directory
>>
>> ** Changed in: python-mode
>> Status: Fix Committed => In Progress
>>
>
>did you re-compile before loading?

I deleted the entire python-mode directory and downloaded from scratch.

as py-temp-directory is set by defvar, you might need to restart Emacs too.

Does the bug still occur?

I always restart Emacs, just to be sure.

Sent from my mobile device

On Apr 7, 2012, at 3:50 AM, "Andreas Roehler" <email address hidden> wrote:

> as py-temp-directory is set by defvar, you might need to restart Emacs
> too.
>
> Does the bug still occur?
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/958987
>
> Title:
> temp file stored in python script directory?
>
> Status in An Emacs mode for editing Python code:
> In Progress
>
> Bug description:
> Running python-mode revision 900 under Windows, when I execute a
> script using C-c C-c, I see a message in the minibuffer like
>
> Wrote c:/python27/Scripts/ipython.bat11144y6h.py
>
> Now this is a lot better functionally than some earlier revisions
> (when the file couldn't be saved at all), but is this a good
> directory to be saving temp files?
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/python-mode/+bug/958987/+subscriptions

what does C-h v py-custom-temp-directory say?

> what does C-h v py-custom-temp-directory say?

py-custom-temp-directory is a variable defined in `python-mode.el'.
Its value is "c:/temp"
Original value was ""

(Directory c:/temp does exist on my machine)

A quick follow-up with the results of some experiments using r927 under Windows (Unfortunately, r928 breaks things too much to run the experiments at all...):

1) If I press C-c C-c while visiting the following file:

#!/usr/bin/ipython
print "Hi, Richard"

I get the message

Wrote c:/temp/ipython9724znA.py

So the temp directory code works fine here.

2) If instead I use the same file but without the shebang line,

print "Hi, Richard"

I get the message

Wrote c:/python27/Scripts/ipython.bat9724AyG.py

So the temp file is written in the python directory in this case.

In both cases, py-shell-name was set to "c:/python27/Scripts/ipython.bat"

3) If I rerun experiment 2, but with py-shell-name set to "ipython", I get the following message:

Wrote c:/temp/ipython9724N8M.py

So the problem only seems to occur when python-mode things it's using a non-default version of ipython.

I've done some more exploring. The problem is caused by the function make-temp-name, called at line 7243 in r927.

Run as in experiment 2 above, make-temp-name returns "c:/python27/Scripts/ipython.bat8616nTl", which is assigned to the variable temp.

In the next line, line 7244,

(expand-file-name temp py-temp-directory)

is executed with py-temp-directory correctly set to "c:/temp", and returns "c:/python27/Scripts/ipython.bat8616nTl".

That's where the problem occurs. I suspect the problem would go away if make-temp-name just stripped out all the ":" and "/" characters in the shell name (or replaced them with "!" characters, say).

I've just verified my conjecture at the end of the last message. I patched python-mode.el (r927) as follows:

--- python-mode.el.r927 2012-04-08 21:04:09 -0700
+++ python-mode.el 2012-04-08 21:37:04 -0700
@@ -7240,7 +7240,9 @@
          (sepchar (or sepchar (py-separator-char)))
          ;; (name-raw (or shell (py-choose-shell)))
          (name (py-buffer-name-prepare shell sepchar))
- (temp (make-temp-name shell))
+ (temp (replace-regexp-in-string ":" "!"
+ (replace-regexp-in-string
+ "/" "!" (make-temp-name shell))))
          (file (concat (expand-file-name temp py-temp-directory) ".py"))
          (filebuf (get-buffer-create file))
          ;; (process-connection-type 'pty)

Now when I run experiment 2, I get the following message:

Wrote c:/temp/c!!python27!Scripts!ipython.bat12188rll.py

Not maybe the most elegant solution, but at least it works.

please re-open if not done

Almost... Replacing sepchar with "-" gets most of the way there, but under Windows you also need to replace the character ":". This may show up in the buffer name (e.g., *ND c:/python27/Scripts/IPython.bat*), but is illegal inside a Windows file name, causing python-mode (r934) to think it's saved a temp file, but actually it's saved a file called "c".

Am 20.03.2012 20:59, schrieb Richard Stanton:
>>> 1) The directory is not being applied correctly.
>>>
>>
>> need a recipe for it, a new report would help, as it's related only
>
> Actually, I think this is the original problem I was reporting. Any
> python script will do in my experience, but the problem only occurs if
> the script file does *not* contain a shebang line. For example, the
> following file works fine:
>
> --------
>
> #!/usr/bin/python
> print "Hi, Richard"
>
> ---------
>
> In this case, the temp file is created in the appropriate temp
> directory.
>
> However, if I use the same file without the shebang line,
>
> --------
>
> print "Hi, Richard"
>
> ---------
>
> now the problem occurs.
>

hmm, let me have a look at the shell-output in both cases - from prompt to prompt
thanks

>>
>>#!/usr/bin/python
>> print "Hi, Richard"
>>
>> ---------
>>
>> In this case, the temp file is created in the appropriate temp
>> directory.
>>
>> However, if I use the same file without the shebang line,
>>
>> --------
>>
>> print "Hi, Richard"
>>
>> ---------
>>
>> now the problem occurs.
>>
>
>hmm, let me have a look at the shell-output in both cases - from prompt
>to prompt
>Thanks

I think the issue is the name of the python shell buffer in the two cases.
In one case it contains directory separators and in the other it doesn't.

Am 12.04.2012 21:58, schrieb Richard Stanton:
>>>
>>> #!/usr/bin/python
>>> print "Hi, Richard"
>>>
>>> ---------
>>>
>>> In this case, the temp file is created in the appropriate temp
>>> directory.
>>>
>>> However, if I use the same file without the shebang line,
>>>
>>> --------
>>>
>>> print "Hi, Richard"
>>>
>>> ---------
>>>
>>> now the problem occurs.
>>>
>>
>> hmm, let me have a look at the shell-output in both cases - from prompt
>> to prompt
>> Thanks
>
> I think the issue is the name of the python shell buffer in the two cases.
> In one case it contains directory separators and in the other it doesn't.
>

must see what your shells told...

please re-open if not done