Merge lp:~therp-nl/anybox.recipe.openerp/jbaudoux-relative_paths_explicit_jailroot_support into lp:~jbaudoux/anybox.recipe.openerp/20130908_relative_paths

[Stefan Rijnhart (Opener)]

This branch allows the creation of a buildout that can be moved to a jail root and run OpenERP in that jail root without any modifications to any of the files in the buildout.

Please notice the target branch: this proposal is a continuation of the work by Jacques-Etienne Baudoux.

- Apply the use of _relativitize in recent changes in the target branch.
- Add explicit support for jail root in the buildout.cfg [openerp] section, which will update the addons-paths.
- Also allow specification of a portable python executable or the python executable in the jail root, to be put in the generated scripts' she-bangs.

For clarity, an update of the original branch by mr. Baudoux with the series branch of the project has been delegated to the prerequisite branch. I'll leave that branch in WIP state, as I don't believe it really needs a separate review.

[lp:~therp-nl/anybox.recipe.openerp/jbaudoux-relative_paths_explicit_jailroot_support]

429. By Stefan Rijnhart (Opener) on 2014-02-06

[FIX] Terminology

430. By Stefan Rijnhart (Opener) on 2014-02-10

[FIX] Remove call to debugger

[Jacques-Etienne Baudoux (jbaudoux)]

Hello Stefan,

I tried your latest branch lp:~therp-nl/anybox.recipe.openerp/8.2-jbaudoux-relative_paths_explicit_jailroot_support to test the jailroot.
It's not clear for me what you expect in the new "jailroot-buildout-dir" option.

When instanciating Session, it's not clear to me what is expected as root path. Seems only used to find a VERSION.txt file that I can't find anyway.
| "session = Session(%s, %r)" % (
|| self._relativitize(self.config_path),
|| self.jailroot_buildout_dir or self.buildout_dir),

When deploying from /root into /root/jailroot, do you expect jailroot-buildout-dir to be '/' which is the root from inside the jail, or '/root/jailroot' which is the path of the jail.
In the later case, that path was already deducted in the method _relativitize in server.py without having to define a new variable.

At the moment, your changes do not work with my current setup. If you can clarify your approach, I'm willing to refactor your MP to make it work with my setup against latest 1.8 serie. I'd be happy to finalize this branch and get it merged.

review: Needs Fixing (test & code review)

Unmerged revisions

430. By Stefan Rijnhart (Opener) on 2014-02-10

[FIX] Remove call to debugger

429. By Stefan Rijnhart (Opener) on 2014-02-06

[FIX] Terminology

428. By Stefan Rijnhart (Opener) on 2014-02-06

[IMP] Comment

427. By Stefan Rijnhart (Opener) on 2014-02-06

[FIX] Typo

426. By Stefan Rijnhart (Opener) on 2014-02-06

[RFR] Terminology

425. By Stefan Rijnhart (Opener) on 2014-02-06

[RFR] Make _relative_path available in both base.py and server.py
[ADD] Adapt addons paths to explicit jail root location
[ADD] Option to specify target python executable

424. By Stefan Rijnhart (Opener) on 2014-02-05

[MRG] Update with target branch

=== modified file 'anybox/recipe/openerp/base.py'
--- anybox/recipe/openerp/base.py	2014-02-10 12:13:18 +0000
+++ anybox/recipe/openerp/base.py	2014-02-10 12:13:18 +0000
@@ -35,6 +35,21 @@
     rfc822.mktime_tz(rfc822.parsedate_tz(h))
 
 
+def _relative_path(common, path):
+    """Copied from easy_install"""
+    r = []
+    while 1:
+        dirname, basename = os.path.split(path)
+        r.append(basename)
+        if dirname == common:
+            break
+        if dirname == path:
+            raise AssertionError("dirname of %s is the same" % dirname)
+        path = dirname
+    r.reverse()
+    return os.path.join(*r)
+
+
 class MainSoftware(object):
     """Placeholder to represent the main software instead of an addon location.
 
@@ -124,6 +139,12 @@
         self.vcs_clear_locks = clear_locks == 'true'
         clear_retry = options.get('vcs-clear-retry', '').lower()
         self.clear_retry = clear_retry == 'true'
+        self.jailroot_buildout_dir = options.get('jailroot-buildout-dir')
+        self.python_scripts_executable = options.get(
+            'python-scripts-executable')
+
+        if self.jailroot_buildout_dir:
+            options['relative-paths'] = 'true'
 
         # same as in zc.recipe.eggs
         relative_paths = options.get(
@@ -521,6 +542,18 @@
             self.sources[local_path] = ((source[0], (source[1][0], revision))
                                         + source[2:])
 
+    def rewrite_addons_path(self, addons_path):
+        """
+        Rewrite the addons path if we are preparing a buildout
+        for a jail root
+        """
+        if self.jailroot_buildout_dir:
+            relative_addons_path = _relative_path(
+                self._relative_paths, addons_path)
+            return os.path.join(
+                self.jailroot_buildout_dir, relative_addons_path)
+        return addons_path
+
     def retrieve_addons(self):
         """Peform all lookup and downloads specified in :attr:`sources`.
 
@@ -1166,6 +1199,11 @@
                 assert os.path.isdir(path), (
                     "Not a directory: %r (aborting)" % path)
 
+        addons_paths = list(self.addons_paths)
+        self.addons_paths = [
+            self.rewrite_addons_path(addons_path)
+            for addons_path in addons_paths
+            ]
         self.options['options.addons_path'] = ','.join(self.addons_paths)
         if self.major_version <= (6, 0):
             self._60_fix_root_path()
 
=== modified file 'anybox/recipe/openerp/server.py'
--- anybox/recipe/openerp/server.py	2014-02-10 12:13:18 +0000
+++ anybox/recipe/openerp/server.py	2014-02-10 12:13:18 +0000
@@ -7,28 +7,13 @@
 import subprocess
 import zc.buildout
 from anybox.recipe.openerp import devtools
-from anybox.recipe.openerp.base import BaseRecipe
+from anybox.recipe.openerp.base import BaseRecipe, _relative_path
 
 logger = logging.getLogger(__name__)
 
 SERVER_COMMA_LIST_OPTIONS = ('log_handler', )
 
 
-def _relative_path(common, path):
-    """Copied from easy_install"""
-    r = []
-    while 1:
-        dirname, basename = os.path.split(path)
-        r.append(basename)
-        if dirname == common:
-            break
-        if dirname == path:
-            raise AssertionError("dirname of %s is the same" % dirname)
-        path = dirname
-    r.reverse()
-    return os.path.join(*r)
-
-
 class ServerRecipe(BaseRecipe):
     """Recipe for server install and config
     """
@@ -69,6 +54,17 @@
                                 "https://launchpad.net/openerp-command."),
         })
 
+        if self.python_scripts_executable:
+            # Monkeypatch the script headers to replace the python
+            # executable by the one configured by the user
+            new_header = '#!%s' % self.python_scripts_executable
+            zc.buildout.easy_install.script_template = (
+                zc.buildout.easy_install.script_template.replace(
+                    zc.buildout.easy_install.script_header, new_header))
+            zc.buildout.easy_install.py_script_template = (
+                zc.buildout.easy_install.py_script_template.replace(
+                    zc.buildout.easy_install.script_header, new_header))
+
     def apply_version_dependent_decisions(self):
         """Store some booleans depending on detected version.
 
@@ -358,9 +354,10 @@
         script_source_path = self.make_absolute(script[0])
         desc.update(
             entry='openerp_upgrader',
-            arguments='%r, %r, %r, %r' % (
-                script_source_path, script[1],
-                self.config_path, self.buildout_dir),
+            arguments='%s, %r, %s, %r' % (
+                self._relativitize(script_source_path), script[1],
+                self._relativitize(self.config_path),
+                self.jailroot_buildout_dir or self.buildout_dir),
         )
 
         if not os.path.exists(script_source_path):
@@ -480,7 +477,7 @@
             "from anybox.recipe.openerp.runtime.session import Session",
             "session = Session(%s, %r)" % (
                     self._relativitize(self.config_path),
-                    self.buildout_dir),
+                    self.jailroot_buildout_dir or self.buildout_dir),
             "if len(sys.argv) <= 1:",
             "    print('To start the OpenERP working session, just do:')",
             "    print('    session.open(db=DATABASE_NAME)')",
@@ -520,8 +517,9 @@
         common_init = os.linesep.join((
             "",
             "from anybox.recipe.openerp.runtime.session import Session",
-            "session = Session(%r, %r)" % (self.config_path,
-                                           self.buildout_dir),
+            "session = Session(%s, %r)" % (
+                    self._relativitize(self.config_path),
+                    self.jailroot_buildout_dir or self.buildout_dir),
         ))
 
         for script_name, desc in self.openerp_scripts.items():
 
=== modified file 'doc/configuration.rst'
--- doc/configuration.rst	2014-02-10 12:13:18 +0000
+++ doc/configuration.rst	2014-02-10 12:13:18 +0000
@@ -320,6 +320,39 @@
 
 Note that tarball downloads get re-extracted afresh in any case.
 
+:: _relocation_options:
+
+Options for buildout relocation and jail root
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. _relative-paths:
+
+relative-paths
+--------------
+
+If set to true, this option will set up a movable buildout. Paths in scripts
+will be made relative to the buildout directory so that it can be relocated
+on the file system.
+
+.. _jailroot-buildout-dir:
+
+jailroot-buildout-dir
+---------------------
+
+If you will be moving this buildout to a jail root, specify the location of
+the buildout in the jail root. The addons-paths in the OpenERP configuration
+file will be adapted to this path. This option implies :ref:`relative-paths`.
+
+.. _python-scripts-executable:
+
+python-scripts-executable
+-------------------------
+
+Specify an alternative Python executable to be used in the she-bang of
+generated scripts. For use in a jail root, ``/usr/bin/env python`` should
+be a sane value, or you could hardcode the path to the Python executable
+in your jail root.
+
 .. _openerp_options:
 
 OpenERP options