Merge lp:~achipa/web2py/cron into lp:~mdipierro/web2py/devel

[Attila Csipa]

[Attila Csipa (achipa)]

wsgihooks fix and examples docs

[lp:~achipa/web2py/cron]

559. By Massimo Di Pierro <email address hidden> on 2009-02-27

modified gaehandler to allow subdirs, thank you Markus

560. By Massimo Di Pierro <email address hidden> on 2009-02-28

-p and -i no longer ignored by widget

561. By Massimo Di Pierro <email address hidden> on 2009-03-01

added __init__.py, thank you Markus

562. By Massimo Di Pierro <email address hidden> on 2009-03-01

pool_size, thanks Markus

563. By Massimo Di Pierro <email address hidden> on 2009-03-01

Edit keep alive and ajax save with feedback, thanks Boris Manojlovic

564. By Massimo Di Pierro <email address hidden> on 2009-03-01

Edit keep alive and ajax save with feedback, thanks Boris Manojlovic

565. By Massimo Di Pierro <email address hidden> on 2009-03-01

force closing connections, thanks Achipa

[Massimo (mdipierro)]

posted in revision 567

review: Approve

[lp:~achipa/web2py/cron]

566. By Massimo Di Pierro <email address hidden> on 2009-03-01

achipa cron patch

567. By Massimo Di Pierro <email address hidden> on 2009-03-01

achipa cron patch

[Massimo (mdipierro)]

this is in

On Feb 27, 2009, at 6:15 AM, AttilaCsipa wrote:

> AttilaCsipa has proposed merging lp:~achipa/web2py/cron into
> lp:~mdipierro/web2py/devel.
>
> Requested reviews:
> Massimo (mdipierro)
>
> wsgihooks fix and examples docs
> --
> https://code.launchpad.net/~achipa/web2py/cron/+merge/4011
> You are requested to review the proposed merge of lp:~achipa/web2py/
> cron into lp:~mdipierro/web2py/devel.
> <review.diff>

=== modified file 'applications/examples/controllers/default.py'
--- applications/examples/controllers/default.py	2009-02-20 05:13:24 +0000
+++ applications/examples/controllers/default.py	2009-02-27 12:05:24 +0000
@@ -55,3 +55,6 @@
 
 def security():
    return response.render(dict())
+
+def cron():
+   return response.render(dict())

=== added file 'applications/examples/views/default/cron.html'
--- applications/examples/views/default/cron.html	1970-01-01 00:00:00 +0000
+++ applications/examples/views/default/cron.html	2009-02-27 12:05:24 +0000
@@ -0,0 +1,87 @@
+{{extend 'layout.html'}}
+{{import os}}
+
+<h1>web2py<sup style="font-size:0.5em;">TM</sup> cron</h1>
+
+The web2py cron provides the ability for applications to execute tasks at preset times, in a platform independent manner (tested on Windows XP, Linux and MacOS X, but should work on any python 2.5 supported platform). 
+
+Cron functionality is defined by a crontab file (in regular <A href="http://en.wikipedia.org/wiki/Cron#crontab_syntax">crontab syntax</A>) in your application's cron directory. This also means that every application can have a separate cron configuration and that cron config can be changed from within web2py without affecting the host OS itself.
+<BR>
+<BR>
+Example:
+{{=CODE(""" 0-59/1  *       *       *       *       root python /path/to/python/script.py
+*/30    *       *       *       *       root *applications/admin/cron/expire_sessions.py
+-1      *       *       *       *       root *mycontroller/myfunction""")}}
+
+As you might have noticed, some of the lines use <A href="#extensions">extensions</A> to regular cron syntax to provide additional web2py functionality.
+
+<BR>
+<h2>Modes of use</h2>
+
+Depending on how you are invoking web2py, there are four modes of operation for web2py cron.
+
+<UL>
+<LI><A href="#soft">Soft cron</A>, available under all execution modes</LI>
+<LI><A href="#hard">Hard cron</A>, available if using the built-in web server (either directly or via Apache mod_proxy)</LI>
+<LI><A href="#external">External cron</A>, available if you have access to the systems own cron service</LI>
+<LI><A href="#no">No cron</A></LI>
+</UL>
+
+The default is hard cron if you are using the built-in web server, in all other cases the default is soft-cron.
+
+
+<h3 id="soft">Soft cron</h3>
+
+Soft cron is the default if you are using CGI, FASTCGI or WSGI. Your tasks will be executed in the first call (page load) to web2py after the time specified in crontab (but AFTER processing the page, so no delay to the user is visible). Obviously, there is some uncertainty exactly when the task will be executed depending on the traffic the site receives. Also, the cron task may get interrupted if the web server has a page load timeout set. If these limitations are not acceptable, see <A href="#external">external cron</A>. Soft cron is a reasonable last resort, but if your web server allows other cron methods, they should be preferred over soft cron.
+
+<h3 id="hard">Hard cron</h3>
+
+Hard cron is the default if you are using the built-in web server (either directly or via Apache mod_proxy). Hard cron is executed in a parallel thread, so unlike soft cron there are no limitations with regard to run time or execution time precision.
+
+<h3 id="external">External cron</h3>
+
+External cron is not default in any scenarios, but requires you to have access to the system cron facilities. It runs in a parallel process, so none of the limitations of soft cron apply. This is the recommended way of using cron under WSGI or FASTCGI. 
+<BR>
+<BR>
+
+Example of line to add to the system crontab, (usually /etc/crontab):
+{{=CODE("""0-59/1 * * * * web2py cd /var/www/web2py/ && touch applications/admin/cron/cron.master && python web2py.py -C -D 1 >> /tmp/cron.output 2>&1""")}}
+
+If you are running external cron, make sure you add the -N command line parameter to your web2py startup script or config so there is no collision of multiple types of cron.
+
+<h3 id="no">No cron</h3>
+
+In case you do not need any cron functionality, you can use the -N command line parameter to disable it. Note that this will disable some maintenance tasks (like the automatic cleaning of session dirs).
+
+<BR>
+<h2 id="extensions">Cron extensions</h2>
+
+Web2py cron has a some extra syntax to support web2py application specifics.
+
+<h3>Calling scripts in the web2py environment</h3>
+
+If the task/script is prefixed with an asterisk and ends with ".py", it will be executed in the web2py environment. This means you will have all the controllers and models at your disposal. Warning: be careful how you use models. While the execution happens in a separate process, database locks have to be taken into account in order to avoid pages waiting for cron tasks that be blocking the database.
+
+<BR>
+<BR>
+Example:
+{{=CODE("""*/30    *       *       *       *       root *applications/admin/cron/expire_sessions.py""")}}
+
+<h3>Calling controller functions</h3>
+
+Same as above, but a function from a controller is executed instead of a separate script file.
+<BR>
+<BR>
+
+Example:
+{{=CODE("""*/30    *       *       *       *       root *mycontroller/myfunction""")}}
+
+<h3>Application initialization</h3>
+
+If you specify -1 as minutes in the crontab file, the given task will be executed only ONCE, on web2py startup. You can use this feature if you want to precache, check or initialize data for an application on web2py startup.
+
+Example:
+{{=CODE("""
+-1      *       *       *       *       root *mycontroller/myfunction
+""")}}
+

=== modified file 'applications/examples/views/default/index.html'
--- applications/examples/views/default/index.html	2009-02-20 05:13:24 +0000
+++ applications/examples/views/default/index.html	2009-02-27 12:05:24 +0000
@@ -32,7 +32,7 @@
 <li>Includes <a href="http://www.jquery.com">jQuery</a> for Effects and Ajax</li>
 <li>Includes a powerful Python-based template language to render objects into HTML or other text-based protocols</li>
 <li>Handles automatically upload and download <a href="{{=URL(r=request,f='examples#example_stream')}}">streaming</a> for large media files</li>
-<li>Includes portable cron mechanism for creating and running recurrent tasks in background</li>
+<li>Includes portable <A href="{{=URL(r=request,f='cron')}}">cron</A> mechanism for creating and running recurrent tasks in background</li>
 <li><a href="{{=URL(r=request,f='license')}}">Allows</a> to bytecode-compile applications and distribute them in closed source</li>
 <li>We guarantee backward compatibility of the web2py API since 1.0</li>
 </ul>

=== modified file 'gluon/contrib/wsgihooks.py'
--- gluon/contrib/wsgihooks.py	2009-02-03 17:38:58 +0000
+++ gluon/contrib/wsgihooks.py	2009-02-27 11:18:33 +0000
@@ -47,7 +47,7 @@
         try:
             result = self.__application(environ, start_response)
         except:
-            self.__callback(self.__environ)
+            self.__callback(environ)
             raise
         return Generator2(result, self.__callback, environ)
 
@@ -63,17 +63,6 @@
         threading.Thread.__init__(self)
         self.env = env
 
-
-#    def run():
-#        if self.func:
-#            return self.func
-
-# class PreConnectionTask(threading.Thread):
-#    def run(self):
-#        import logging
-#        logging.warning("pre")
-
-
 def callback(env):
     global globalposttasks
     global localposttasks