Merge lp:~camptocamp/magentoerpconnect/oerp61-oldstable-import-partners-2012-12-13 into lp:magentoerpconnect/oerp6.1-oldstable

[Yannick Vaucher @ Camptocamp]

Implement the logic for global partner import

Plus add a check on SO import from magento to make sure a partner having the same email is updated before importing SO emitter partner (in case of an outdated and forgot account of the same customer to solve)

More detail on bug report:
lp:1083592

Needs https://code.launchpad.net/~c2c/openobject-addons/extra_trunk-add-import-partners-2012-12-13/+merge/139731

[Guewen Baconnier @ Camptocamp (gbaconnier-c2c)]

you may want to remove the FIXME tag at line 31 as you have fixed the issue ;-)

@nitpicking
l89. missing blank line
l93, l111 extra blank lines
The pep8 convention specifies 2 lines before classes and top level-functions and 1 single line between methods in class definition [0]

l112 'import_partners' should preferably be '_import_partners' because it can't be called from the outside (with XML/RPC for instance) due to the browse_record argument.

l158 useless parenthesis

l187 if website_id:
I don't think that it is possible to have a website_id which is 0, but if it could be the case we would preferably test if website is not None:

l189 mini typo s/an other/another/

l200
from A to B on a old account and create a new account with B address.
shouldn't it be?
from A to B on a old account and create a new account with A address.

Apart of these details (not all of them need to be changed, but please check the comment at l200), it sounds good to me, but I'll review the corresponding merge in extra-trunk and come back here.

[0] http://www.python.org/dev/peps/pep-0008/#blank-lines

[Guewen Baconnier @ Camptocamp (gbaconnier-c2c)]

l108 it must filter on the website

review: Needs Fixing

[Yannick Vaucher @ Camptocamp (yvaucher-c2c)]

l187 if website_id:
I don't think that it is possible to have a website_id which is 0, but if it could be the case we would preferably test if website is not None:

If a website_id could be 0, in that case, website_id is at first a string before we cast it to int.
It could be '0' but not 0

Seems fine to me like that

I will fix the rest ASAP

[Yannick Vaucher @ Camptocamp (yvaucher-c2c)]

Filter added on website if website_id is defined.

[Guewen Baconnier @ Camptocamp (gbaconnier-c2c)]

Yay, seems great now.
Thanks

review: Approve

[Guewen Baconnier @ Camptocamp (gbaconnier-c2c)]

Just noticed that the cron is missing, can you add it please?
Thanks

review: Needs Fixing

[lp:~camptocamp/magentoerpconnect/oerp61-oldstable-import-partners-2012-12-13]

668. By Yannick Vaucher @ Camptocamp on 2013-01-09

[ADD] magentoerpconnect - cron job for partner import

[Yannick Vaucher @ Camptocamp (yvaucher-c2c)]

Here we go for the cron job

[Guewen Baconnier @ Camptocamp (gbaconnier-c2c)]

I fixed one thing during the merge: run_import_partners_scheduler should not return a None value.

[Guewen Baconnier @ Camptocamp (gbaconnier-c2c)]

review: Approve

=== modified file 'magentoerpconnect/magerp_data.xml'
--- magentoerpconnect/magerp_data.xml	2011-11-15 15:42:00 +0000
+++ magentoerpconnect/magerp_data.xml	2013-01-09 15:14:26 +0000
@@ -1,36 +1,36 @@
 <?xml version="1.0"?>
 <openerp>
-	<data noupdate="1">
-		
-		<record id="product_product_cash_on_delivery" model="product.product">
-			<field name="default_code">CASH ON DELIVERY MAGENTO</field>
-			<field name="list_price">0.0</field>
-			<field name="standard_price">0.0</field>
-			<field name="type">service</field>
-			<field name="name">Cash on delivery</field>
-			<field name="magento_exportable" eval="False" />
-			<field name="categ_id" ref="base_sale_multichannels.categ_0"/>
-		</record>
-
-		<record id="product_product_gift" model="product.product">
-			<field name="default_code">GIFT CERTIFICATE</field>
-			<field name="list_price">0.0</field>
-			<field name="standard_price">0.0</field>
-			<field name="type">service</field>
-			<field name="name">Gift Certificate</field>
-			<field name="magento_exportable" eval="False" />
-			<field name="categ_id" ref="base_sale_multichannels.categ_0"/>
-		</record>
-
-		<record id="product_product_discount" model="product.product">
-			<field name="default_code">DISCOUNT MAGENTO</field>
-			<field name="list_price">0.0</field>
-			<field name="standard_price">0.0</field>
-			<field name="type">service</field>
-			<field name="name">Discount Coupon</field>
-			<field name="magento_exportable" eval="False" />
-			<field name="categ_id" ref="base_sale_multichannels.categ_0"/>
-		</record>
+    <data noupdate="1">
+
+        <record id="product_product_cash_on_delivery" model="product.product">
+            <field name="default_code">CASH ON DELIVERY MAGENTO</field>
+            <field name="list_price">0.0</field>
+            <field name="standard_price">0.0</field>
+            <field name="type">service</field>
+            <field name="name">Cash on delivery</field>
+            <field name="magento_exportable" eval="False" />
+            <field name="categ_id" ref="base_sale_multichannels.categ_0"/>
+        </record>
+
+        <record id="product_product_gift" model="product.product">
+            <field name="default_code">GIFT CERTIFICATE</field>
+            <field name="list_price">0.0</field>
+            <field name="standard_price">0.0</field>
+            <field name="type">service</field>
+            <field name="name">Gift Certificate</field>
+            <field name="magento_exportable" eval="False" />
+            <field name="categ_id" ref="base_sale_multichannels.categ_0"/>
+        </record>
+
+        <record id="product_product_discount" model="product.product">
+            <field name="default_code">DISCOUNT MAGENTO</field>
+            <field name="list_price">0.0</field>
+            <field name="standard_price">0.0</field>
+            <field name="type">service</field>
+            <field name="name">Discount Coupon</field>
+            <field name="magento_exportable" eval="False" />
+            <field name="categ_id" ref="base_sale_multichannels.categ_0"/>
+        </record>
 
         <record forcecreate="True" id="ir_cron_import_orders_scheduler_action" model="ir.cron">
             <field name="name">Magento Import Orders</field>
@@ -136,15 +136,29 @@
             <field eval="'()'" name="args"/>
         </record>
 
+        <record forcecreate="True" id="ir_cron_import_partners_scheduler_action" model="ir.cron">
+            <field name="name">Magento Import Partners</field>
+            <field eval="False" name="active"/>
+            <field name="user_id" ref="base.user_root"/>
+            <field name="interval_number">1</field>
+            <field name="interval_type">days</field>
+            <field name="numbercall">-1</field>
+            <field eval="False" name="doall"/>
+            <field eval="'external.shop.group'" name="model"/>
+            <field eval="'run_import_partners_scheduler'" name="function"/>
+            <field eval="'()'" name="args"/>
+        </record>
+
         <record id="payment_type1" model="base.sale.payment.type">
-			<field name="name">checkmo;cashondelivery</field>
-			<field name="order_policy">manual</field>
-		</record>
-		
+            <field name="name">checkmo;cashondelivery</field>
+            <field name="order_policy">manual</field>
+        </record>
+
         <record id="payment_type2" model="base.sale.payment.type">
-			<field name="name">ccsave;free;googlecheckout;paypayl_express;paybox_system;paypal_standard;servired_standard;bbva;cofidis</field>
-			<field name="order_policy">prepaid</field>
-		</record>
-
-	</data>
+            <field name="name">ccsave;free;googlecheckout;paypayl_express;paybox_system;paypal_standard;servired_standard;bbva;cofidis</field>
+            <field name="order_policy">prepaid</field>
+        </record>
+
+
+    </data>
 </openerp>
 
=== modified file 'magentoerpconnect/partner.py'
--- magentoerpconnect/partner.py	2012-05-16 09:15:39 +0000
+++ magentoerpconnect/partner.py	2013-01-09 15:14:26 +0000
@@ -317,4 +317,69 @@
 
         return False, partner_id
 
+    def _ext_import_one(self, cr, uid, external_id, vals, external_data, referential_id, defaults=None, context=None):
+        """ Import one external resource
+
+        This method can be inherited to do an action which have to be done
+        within the same transaction than the import of the resource.
+        All the database operations done within _ext_import_one
+        will be rollbacked if any error occurs.
+        So no action on an external referential should be done within this method.
+
+        :param int external_id: id of the resource on the external referential
+        :param dict vals: vals converted to openerp
+        :param dict external_data: vals of the external resource before conversion
+        :param int referential_id: external referential id from where we import the resource
+        :param dict defaults: defaults value for fields which are not in vals
+        :return: tuple created id, updated id
+        """
+        cid, wid = super(res_partner, self)._ext_import_one(
+                cr, uid, external_id, vals, external_data, referential_id, defaults=defaults, context=context)
+
+        partner_id = cid or wid
+        if partner_id:
+            partner = self.browse(cr, uid, partner_id, context=context)
+
+            # As a partner without addresses can't be opened
+            # we create a dummy empty address
+            if not partner.address:
+                new_empty_address = [(0, 0, {})]
+                partner.write({'address': new_empty_address}, context=context)
+
+        return cid, wid
+
+    def _import_partner_from_external(
+            self,
+            cr, uid,
+            partner_ext_id,
+            connection,
+            referential_id,
+            context=None):
+        """
+        Import partner data from magento for a given id
+
+        :param partner_ext_id: magento id of customer to import
+        :param connection: connection data to magento
+        :param referential_id: referential id from where we import the ressource
+        """
+        if context is None:
+            context = {}
+        imp_ctx = dict(context,
+                       id=partner_ext_id)
+        result = self.get_external_data(
+            cr, uid,
+            connection,
+            referential_id,
+            defaults={},
+            context=imp_ctx)
+        partner_id = self.extid_to_oeid(
+            cr, uid, partner_ext_id, referential_id, context=context)
+
+        # update the address book of the customer, this addresses
+        # will be available in the partner form and the searches
+        self.import_magento_address_book(
+            cr, uid, partner_id, referential_id, context=context)
+
+        return result
+
 res_partner()
 
=== modified file 'magentoerpconnect/sale.py'
--- magentoerpconnect/sale.py	2012-12-06 08:08:36 +0000
+++ magentoerpconnect/sale.py	2013-01-09 15:14:26 +0000
@@ -30,6 +30,7 @@
 from tools.translate import _
 import string
 #from datetime import datetime
+from contextlib import closing
 import tools
 import time
 from tools import DEFAULT_SERVER_DATETIME_FORMAT
@@ -49,6 +50,113 @@
     'waiting_date': 'holded'}
 SALE_ORDER_IMPORT_STEP = 200
 
+
+class external_shop_group(magerp_osv.magerp_osv):
+    _inherit = 'external.shop.group'
+
+    @staticmethod
+    def _get_magento_partners(connection, from_date=False, website_id=False):
+        """
+        Get data from magento of new and updated customers since a specific_date
+
+        :param connection: connection data to magento
+        :param from_data: date from which we get the changes
+        """
+        filters = []
+        if from_date:
+            filters = [
+                    {'created_at': {'gt': from_date}}, # OR
+                    {'updated_at': {'gt': from_date}}
+                    ]
+        if website_id:
+            if filters:
+                for f in filters:
+                    f.update(website_id={'eq': website_id})
+            else:
+                filters = [{'website_id': {'eq': website_id}}]
+
+        data = connection.call('customer.list', filters)
+        return data
+
+    def _import_partners(self, cr, uid, group, context=None):
+        """
+        Import partners for a single shop group
+
+        :param group: browse record of an external.shop.group
+        """
+        if context is None: context = {}
+
+        referential_id = group.referential_id.id
+        from_date = group.import_partners_from_date
+        website_id = self.oeid_to_extid(cr, uid, group.id, referential_id)
+        partner_obj = self.pool.get('res.partner')
+        result = super(external_shop_group, self)._import_partners(
+                cr, uid, group,
+                context=context)
+
+        ref_obj = self.pool.get('external.referential')
+        connection = context.get('conn_obj') or \
+                        ref_obj.external_connection(
+                            cr, uid, referential_id, context=context)
+
+        [result.setdefault(key, []) for key in ['create_ids', 'write_ids', 'unchanged_ids']]
+
+        # Get partners from magento which where created or updated
+        # since last import
+        data = self._get_magento_partners(connection, from_date, website_id)
+
+        data.sort(key=lambda customer: customer['updated_at'] or customer['created_at'])
+
+        imp_ctx = dict(context, import_no_new_cr=True)
+        with closing(pooler.get_db(cr.dbname).cursor()) as local_cr:
+            # here we commit each partner updated or created as it can
+            # be an long processing and we want to be able to restart
+            # from the state when it failed
+            for customer in data:
+                ext_id = customer['customer_id']
+                change_date = customer['updated_at'] or customer['created_at']
+                try:
+                    current_result = partner_obj._import_partner_from_external(
+                            local_cr, uid, ext_id,
+                            connection, referential_id,
+                            context=imp_ctx)
+                except Exception as e:
+                    local_cr.rollback()
+                    if "res_partner_emailid_uniq" in e.message:
+                        message = _("Cannot import a partner: "
+                                    "A partner with email %s already exists ")\
+                                    % customer['email']
+                        raise osv.except_osv(_("Import Error"), message)
+                    else:
+                        raise
+                else:
+                    self.write(
+                            local_cr, uid,
+                            group.id,
+                            {'import_partners_from_date': change_date},
+                            context=context)
+                    local_cr.commit()
+                    result['create_ids'] += current_result.get('create_ids')
+                    result['write_ids'] += current_result.get('write_ids')
+
+        return result
+
+    def run_import_partners_scheduler(self, cr, uid, context=None):
+        """
+        Methode to launch the import on all external shop groups
+        to be used by a cron job
+        """
+        group_ids = self.search(cr, uid, [], context=context)
+
+        if group_ids:
+            shop_groups = self.browse(cr, uid, ids, context=context)
+
+            for group in shop_groups:
+                self._import_partners(cr, uid, group, context=context)
+
+
+external_shop_group()
+
 class sale_shop(magerp_osv.magerp_osv):
     _inherit = "sale.shop"
 
@@ -340,22 +448,39 @@
                 cr, uid, customer_record, referential_id,
                 defaults=partner_defaults, context=context)
         else:
+            ext_id = data_record['customer_id']
+            connection = context.get('conn_obj')
+            website_id = data_record.get('website_id')
+            if website_id:
+                website_id = int(data_record['website_id'])
+            # check if another customer exists
+            existing_id = partner_obj.search_magento_partner(
+                    cr, uid,
+                    data_record['customer_email'],
+                    website_id,
+                    context=context)
+            if existing_id:
+                # check if the ext_id is the same
+                # to make sure it isn't some outdated data
+                # This could happen when a customer change his email
+                # from A to B on a old account and create
+                # a new account with A address.
+                # in such case we want to make sure the old account is
+                # updated
+                existing_ext_id = partner_obj.oeid_to_extid(cr, uid, existing_id, referential_id)
+                if existing_ext_id != ext_id:
+                    partner_obj._import_partner_from_external(
+                            cr, uid, existing_ext_id,
+                            connection, referential_id,
+                            context=context)
+
             # always update the customer when importing an order
-            imp_ctx = dict(context)
-            imp_ctx['id'] = data_record['customer_id']
-            partner_obj.get_external_data(
-                cr, uid,
-                context.get('conn_obj'),
-                referential_id,
-                defaults={},
-                context=imp_ctx)
-            partner_id = partner_obj.extid_to_oeid(
-                cr, uid, data_record['customer_id'], referential_id)
-
-            # update the address book of the customer, this addresses
-            # will be available in the partner form and the searches
-            partner_obj.import_magento_address_book(
-                cr, uid, partner_id, referential_id, context=context)
+            result = partner_obj._import_partner_from_external(
+                    cr, uid, ext_id,
+                    connection, referential_id,
+                    context=context)
+            partner_id = (result['create_ids'] and result['create_ids'][0]
+                          or result['write_ids'] and result['write_ids'][0])
 
         # The addresses of the sale order are imported as active=false
         # so they are linked with the sale order but they are not displayed