Merge lp:~akretion-team/openupgrade-addons/61-to-70-document into lp:openupgrade-addons

[Alexis de Lattre]

Document module : add migration scripts for 6.1 -> 7.0 migration

[Alexis de Lattre (alexis-via)]

The web page for my branch is broken. I have opened a ticket for the LP support team :

https://answers.launchpad.net/launchpad/+question/249758

I hope that it will be fixed shortly.

[Yann Papouin (yann-papouin)]

The same two or three times, I had to delete my branch from launchpad and re-push it from my local repo.

[Pedro Manuel Baeza (pedro.baeza)]

Hi, Alexis,

In a first sight, pre-migration.py doesn't do anything. You need to add this code:

@openupgrade.migrate()
def migrate(cr, version):
    openupgrade.rename_columns(cr, column_renames)

But I don't see then what do you do with renamed columns to None on post script. Can you enligthened me?

Regards.

review: Needs Information (code review)

[Alexis de Lattre (alexis-via)]

@Pedro

You are right, I declared the rename_columns, and I forgot to use it in the post-migration... but it was working ! For a strange reason, the columns of the document_storage were kept during the upgrade.

But Sylvain convinced me of updating my code, so I updated the code to use the rename_columns :)

[Pedro Manuel Baeza (pedro.baeza)]

Although this migration in most cases will require manual intervertion afterwards, it's the most conservative. Or what do you think about move files at OS level to a fixed path and configure ir.parameter to this fixed path. In this way, we are getting the desired goal of OpenUpgrade to do it all by itself. What do you think about this?

Regards.

review: Needs Information (code review)

[Alexis de Lattre (alexis-via)]

@Pedro

I don't have enough experience on OpenUpgrade to have an opinion on this. This is a question for Stefan or Sylvain I think...

But moving files at OS level requires write permission on the destination directory (and may also require to create the destination directory), and the user who execute the OpenERP server for openupgrade may not have the required permissions.

Also, if we move files to a new directory and this directory is on another partition, we should make sure that there is enough space on the destination partition. All these files may weight several GB...

[Pedro Manuel Baeza (pedro.baeza)]

Yeah, you're right about some considerations. I think write condition can be a must for this, or show a warning about that. Another option can be to make a symbolic link (only for Linux and Windows), and in this way, you don't need to check disk space.

Let's involve as you say another people like Stefan or Sylvain to say their opinion.

Regards.

[Pedro Manuel Baeza (pedro.baeza)]

s/Windows/MacOS

[Sylvain LE GAL (GRAP) (sylvain-legal)]

Hi,

I don't use storage by filesystem for own database for the moment. So I didn't have the problem when I realized my 5.0 -> 6.1 migration and 6.1 -> 7.0.

However, I think that moving files will not work in many cases. As said, depending of OS, space on disk, access right, but mainly because in some cases OpenUpgrade Server is not the production server for some reasons:

1/ You don't have access to the server: Your customer manage his server and send you his database (anonymized or not) to migrate it. (As OpenERP S.A. does)
2/ You have access to the server but prefer to run openupgrade locally or on another server for some reasons.(to do tests, to avoid to install OpenUpgrade on the server because it is a non-production server, etc...)

So The only thing that will allways work is to put a warning in OpenUpgrade log file for the user that upgrade the system.

Regards.

[Pedro Manuel Baeza (pedro.baeza)]

As you both pointed out, there are many restrictions in the move operation, so let's stay as is with the warning.

Regards.

review: Approve (code review)

[Sylvain LE GAL (GRAP) (sylvain-legal)]

Just one thing. If you do not manage the case of the xml_id "dir_partner_category", the document.directory will be deleted ?
Why do you guess that this object is unused ?

What about to do in a pre-migration script a request like the following, to make the object persistent ?

delete * from ir_model_data where module ='document' and name ='dir_partner_category';

Otherwise, all looks good to me.

review: Needs Information (code review, no test.)

[Holger Brunn (Therp) (hbrunn)]

Just for completeness: NTFS (Windows' filesystem) support hard and soft links for years, it's just that there's no default UI for that...

I agree with not touching the files, there's a lot of uncertainties here and the danger to screw up is very high.

review: Approve

=== added file 'document/migrations/7.0.2.1/openupgrade_analysis_work.txt'
--- document/migrations/7.0.2.1/openupgrade_analysis_work.txt	1970-01-01 00:00:00 +0000
+++ document/migrations/7.0.2.1/openupgrade_analysis_work.txt	2014-06-04 22:01:29 +0000
@@ -0,0 +1,72 @@
+---Fields in module 'document'---
+# IMPORTANT NOTE : due to a bug in OpenUpgrade
+# (cf https://github.com/OpenUpgrade/OpenUpgrade/issues/40)
+# one important thing is missing in this analysis file :
+# the table document_storage as been dropped !
+# If you have some storage of type "filestore", openupgrade will add
+# an entry in ir.config_parameter like this one :
+# {key: ir_attachment.location, value: file:///filestore}
+
+# Document.storage has been dropped in v7, so the M2O links to document.storage is removed => nothing to do
+document     / document.directory       / storage_id (many2one)         : DEL relation: document.storage
+
+# These fields are moved to a lower-level module (base), so there is nothing to do.
+document     / ir.attachment            / db_datas (binary)             : module is now 'base' ('document')
+document     / ir.attachment            / file_size (integer)           : module is now 'base' ('document')
+# Note : the size of this char field move upgraded from 200 to 256
+document     / ir.attachment            / store_fname (char)            : module is now 'base' ('document')
+
+# In all my 6.1 production databases, this field is always empty...
+# Looking at the code of the document module in 6.1, it seems that this field
+# was not used
+document     / process.node             / directory_id (many2one)       : DEL relation: document.directory
+
+# report.document.user is _auto = False (SQL view) => nothing to do
+document     / report.document.user     / user (char)                   : now a function
+
+---XML records in module 'document'---
+
+# This record is not provided by default any more, but I think that nobody uses this
+DEL document.directory: document.dir_partner_category
+
+# Document.storage has been dropped in v7
+DEL document.storage: document.storage_default
+
+NEW ir.actions.act_window: document.action_view_files_by_user_graph
+DEL ir.actions.act_window: document.action_document_storage_form
+DEL ir.actions.act_window: document.action_view_files_by_partner
+DEL ir.actions.act_window: document.action_view_user_graph
+DEL ir.actions.act_window: document.action_view_wall
+DEL ir.actions.act_window: document.open_board_document_manager
+DEL ir.actions.act_window: document.open_board_document_manager1
+DEL ir.actions.todo.category: document.category_knowledge_mgmt_config
+DEL ir.model.access: document.access_document_storage_group_document_manager
+DEL ir.model.access: document.access_document_storage_group_system
+DEL ir.model.access: document.access_document_storage_group_user
+DEL ir.model.access: document.access_report_document_wall_group_document
+DEL ir.model.access: document.access_report_document_wall_group_document_manager
+DEL ir.model.access: document.access_report_document_wall_group_system
+DEL ir.model.access: document.access_report_files_partner_group_document
+DEL ir.model.access: document.access_report_files_partner_group_document_manager
+DEL ir.ui.menu: document.menu_document_storage_media
+DEL ir.ui.menu: document.menu_reporting
+DEL ir.ui.menu: document.menu_reports_document
+DEL ir.ui.menu: document.menu_reports_document_manager
+DEL ir.ui.menu: document.menu_reports_document_manager1
+DEL ir.ui.menu: knowledge.menu_document
+NEW ir.ui.view: document.view_files_by_user_graph
+DEL ir.ui.view: document.board_document_manager_form
+DEL ir.ui.view: document.board_document_manager_form1
+DEL ir.ui.view: document.view_attachment_form_inherit
+DEL ir.ui.view: document.view_document_by_resourcetype_graph
+DEL ir.ui.view: document.view_document_storage_filter
+DEL ir.ui.view: document.view_document_storage_form
+DEL ir.ui.view: document.view_document_storage_tree
+DEL ir.ui.view: document.view_document_wall_form
+DEL ir.ui.view: document.view_document_wall_tree
+DEL ir.ui.view: document.view_files_by_partner_form
+DEL ir.ui.view: document.view_files_by_partner_graph
+DEL ir.ui.view: document.view_files_by_partner_tree
+DEL ir.ui.view: document.view_process_form_inherit1
+DEL ir.ui.view: document.view_process_node_form_inherit1
+DEL ir.ui.view: document.view_user_graph

=== added file 'document/migrations/7.0.2.1/post-migration.py'
--- document/migrations/7.0.2.1/post-migration.py	1970-01-01 00:00:00 +0000
+++ document/migrations/7.0.2.1/post-migration.py	2014-06-04 22:01:29 +0000
@@ -0,0 +1,76 @@
+# -*- coding: utf-8 -*-
+##############################################################################
+#
+#    OpenUpgrade, the free software migration tool for OpenERP
+#    Copyright (C) 2014 Akretion (http://www.akretion.com/)
+#    @author: Alexis de Lattre <[e-mail]>
+#
+#    This program is free software: you can redistribute it and/or modify
+#    it under the terms of the GNU Affero General Public License as
+#    published by the Free Software Foundation, either version 3 of the
+#    License, or (at your option) any later version.
+#
+#    This program is distributed in the hope that it will be useful,
+#    but WITHOUT ANY WARRANTY; without even the implied warranty of
+#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#    GNU Affero General Public License for more details.
+#
+#    You should have received a copy of the GNU Affero General Public License
+#    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+##############################################################################
+
+from openerp.openupgrade import openupgrade
+from openerp import pooler, SUPERUSER_ID
+import logging
+
+logger = logging.getLogger('OpenUpgrade')
+
+
+def migrate_filestore(cr, pool):
+    cr.execute(
+        "SELECT %s, %s FROM document_storage WHERE %s='filestore' "
+        "ORDER BY id" % (
+            openupgrade.get_legacy_name('name'),
+            openupgrade.get_legacy_name('path'),
+            openupgrade.get_legacy_name('type'),
+            ))
+    filestore = cr.fetchall()
+    if len(filestore) > 1:
+        logger.error(
+            "You have several filestore, which is very uncommon. OpenUpgrade "
+            "will reconfigure your first filestore (%s), and you will have to "
+            "merge the other filestore into your first filestore !"
+            % filestore[0][0])
+
+    if filestore:
+        logger.info(
+            "Migrating the configuration of the filestore '%s' "
+            "from document_storage to ir.config_parameter" % filestore[0][0])
+        path = filestore[0][1]
+        logger.debug("Path of the first filestore = %s" % path)
+        pool['ir.config_parameter'].create(
+            cr, SUPERUSER_ID, {
+                'key': 'ir_attachment.location',
+                'value': 'file://%s' % path,
+                })
+        logger.warning(
+            "The new configuration of the filestore suppose that it is now "
+            "located at <openerp_server_root>%s. It is not possible to have "
+            "an absolute path any more." % path)
+        logger.warning(
+            "If your filestore is not located at <openerp_server_root>%s, you "
+            "should create this directory and move it there, or update "
+            "the System Parameter that has the key 'ir_attachment.location'."
+            % path)
+        logger.info(
+            "If you want to know more about the filestore in 7.0, read "
+            "the answer from Antony Lesuisse on this question : "
+            "https://accounts.openerp.com/fr_FR/forum/Help-1/question/"
+            "Where-are-document-Attachments-stored-529")
+
+
[e-mail]()
+def migrate(cr, version):
+    pool = pooler.get_pool(cr.dbname)
+    migrate_filestore(cr, pool)

=== added file 'document/migrations/7.0.2.1/pre-migration.py'
--- document/migrations/7.0.2.1/pre-migration.py	1970-01-01 00:00:00 +0000
+++ document/migrations/7.0.2.1/pre-migration.py	2014-06-04 22:01:29 +0000
@@ -0,0 +1,37 @@
+# -*- coding: utf-8 -*-
+##############################################################################
+#
+#    OpenUpgrade, the free software migration tool for OpenERP
+#    Copyright (C) 2014 Akretion (http://www.akretion.com/)
+#    @author: Alexis de Lattre <[e-mail]>
+#
+#    This program is free software: you can redistribute it and/or modify
+#    it under the terms of the GNU Affero General Public License as
+#    published by the Free Software Foundation, either version 3 of the
+#    License, or (at your option) any later version.
+#
+#    This program is distributed in the hope that it will be useful,
+#    but WITHOUT ANY WARRANTY; without even the implied warranty of
+#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+#    GNU Affero General Public License for more details.
+#
+#    You should have received a copy of the GNU Affero General Public License
+#    along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+##############################################################################
+
+from openerp.openupgrade import openupgrade
+
+
+column_renames = {
+    'document_storage': [
+        ('name', None),
+        ('path', None),
+        ('type', None),
+    ]
+    }
+
+
[e-mail]()
+def migrate(cr, version):
+    openupgrade.rename_columns(cr, column_renames)