Merge lp:~camptocamp/magentoerpconnect/oerp61-oldstable-import-partners-2012-12-13 into lp:magentoerpconnect/oerp6.1-oldstable

Proposed by Yannick Vaucher @ Camptocamp on 2012-12-13
Status: Merged
Merged at revision: 665
Proposed branch: lp:~camptocamp/magentoerpconnect/oerp61-oldstable-import-partners-2012-12-13
Merge into: lp:magentoerpconnect/oerp6.1-oldstable
Diff against target: 365 lines (+259/-55)
3 files modified
magentoerpconnect/magerp_data.xml (+54/-40)
magentoerpconnect/partner.py (+65/-0)
magentoerpconnect/sale.py (+140/-15)
To merge this branch: bzr merge lp:~camptocamp/magentoerpconnect/oerp61-oldstable-import-partners-2012-12-13
Reviewer Review Type Date Requested Status
Guewen Baconnier @ Camptocamp 2012-12-13 Approve on 2013-01-10
Review via email: mp+139728@code.launchpad.net

Description of the change

Implement the logic for global partner import

Plus add a check on SO import from magento to make sure a partner having the same email is updated before importing SO emitter partner (in case of an outdated and forgot account of the same customer to solve)

More detail on bug report:
lp:1083592

Needs https://code.launchpad.net/~c2c/openobject-addons/extra_trunk-add-import-partners-2012-12-13/+merge/139731

To post a comment you must log in.

you may want to remove the FIXME tag at line 31 as you have fixed the issue ;-)

@nitpicking
l89. missing blank line
l93, l111 extra blank lines
The pep8 convention specifies 2 lines before classes and top level-functions and 1 single line between methods in class definition [0]

l112 'import_partners' should preferably be '_import_partners' because it can't be called from the outside (with XML/RPC for instance) due to the browse_record argument.

l158 useless parenthesis

l187 if website_id:
I don't think that it is possible to have a website_id which is 0, but if it could be the case we would preferably test if website is not None:

l189 mini typo s/an other/another/

l200
from A to B on a old account and create a new account with B address.
shouldn't it be?
from A to B on a old account and create a new account with A address.

Apart of these details (not all of them need to be changed, but please check the comment at l200), it sounds good to me, but I'll review the corresponding merge in extra-trunk and come back here.

[0] http://www.python.org/dev/peps/pep-0008/#blank-lines

l108 it must filter on the website

review: Needs Fixing

l187 if website_id:
I don't think that it is possible to have a website_id which is 0, but if it could be the case we would preferably test if website is not None:

If a website_id could be 0, in that case, website_id is at first a string before we cast it to int.
It could be '0' but not 0

Seems fine to me like that

I will fix the rest ASAP

Filter added on website if website_id is defined.

Yay, seems great now.
Thanks

review: Approve

Just noticed that the cron is missing, can you add it please?
Thanks

review: Needs Fixing
668. By Yannick Vaucher @ Camptocamp on 2013-01-09

[ADD] magentoerpconnect - cron job for partner import

Here we go for the cron job

I fixed one thing during the merge: run_import_partners_scheduler should not return a None value.

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'magentoerpconnect/magerp_data.xml'
2--- magentoerpconnect/magerp_data.xml 2011-11-15 15:42:00 +0000
3+++ magentoerpconnect/magerp_data.xml 2013-01-09 15:14:26 +0000
4@@ -1,36 +1,36 @@
5 <?xml version="1.0"?>
6 <openerp>
7- <data noupdate="1">
8-
9- <record id="product_product_cash_on_delivery" model="product.product">
10- <field name="default_code">CASH ON DELIVERY MAGENTO</field>
11- <field name="list_price">0.0</field>
12- <field name="standard_price">0.0</field>
13- <field name="type">service</field>
14- <field name="name">Cash on delivery</field>
15- <field name="magento_exportable" eval="False" />
16- <field name="categ_id" ref="base_sale_multichannels.categ_0"/>
17- </record>
18-
19- <record id="product_product_gift" model="product.product">
20- <field name="default_code">GIFT CERTIFICATE</field>
21- <field name="list_price">0.0</field>
22- <field name="standard_price">0.0</field>
23- <field name="type">service</field>
24- <field name="name">Gift Certificate</field>
25- <field name="magento_exportable" eval="False" />
26- <field name="categ_id" ref="base_sale_multichannels.categ_0"/>
27- </record>
28-
29- <record id="product_product_discount" model="product.product">
30- <field name="default_code">DISCOUNT MAGENTO</field>
31- <field name="list_price">0.0</field>
32- <field name="standard_price">0.0</field>
33- <field name="type">service</field>
34- <field name="name">Discount Coupon</field>
35- <field name="magento_exportable" eval="False" />
36- <field name="categ_id" ref="base_sale_multichannels.categ_0"/>
37- </record>
38+ <data noupdate="1">
39+
40+ <record id="product_product_cash_on_delivery" model="product.product">
41+ <field name="default_code">CASH ON DELIVERY MAGENTO</field>
42+ <field name="list_price">0.0</field>
43+ <field name="standard_price">0.0</field>
44+ <field name="type">service</field>
45+ <field name="name">Cash on delivery</field>
46+ <field name="magento_exportable" eval="False" />
47+ <field name="categ_id" ref="base_sale_multichannels.categ_0"/>
48+ </record>
49+
50+ <record id="product_product_gift" model="product.product">
51+ <field name="default_code">GIFT CERTIFICATE</field>
52+ <field name="list_price">0.0</field>
53+ <field name="standard_price">0.0</field>
54+ <field name="type">service</field>
55+ <field name="name">Gift Certificate</field>
56+ <field name="magento_exportable" eval="False" />
57+ <field name="categ_id" ref="base_sale_multichannels.categ_0"/>
58+ </record>
59+
60+ <record id="product_product_discount" model="product.product">
61+ <field name="default_code">DISCOUNT MAGENTO</field>
62+ <field name="list_price">0.0</field>
63+ <field name="standard_price">0.0</field>
64+ <field name="type">service</field>
65+ <field name="name">Discount Coupon</field>
66+ <field name="magento_exportable" eval="False" />
67+ <field name="categ_id" ref="base_sale_multichannels.categ_0"/>
68+ </record>
69
70 <record forcecreate="True" id="ir_cron_import_orders_scheduler_action" model="ir.cron">
71 <field name="name">Magento Import Orders</field>
72@@ -136,15 +136,29 @@
73 <field eval="'()'" name="args"/>
74 </record>
75
76+ <record forcecreate="True" id="ir_cron_import_partners_scheduler_action" model="ir.cron">
77+ <field name="name">Magento Import Partners</field>
78+ <field eval="False" name="active"/>
79+ <field name="user_id" ref="base.user_root"/>
80+ <field name="interval_number">1</field>
81+ <field name="interval_type">days</field>
82+ <field name="numbercall">-1</field>
83+ <field eval="False" name="doall"/>
84+ <field eval="'external.shop.group'" name="model"/>
85+ <field eval="'run_import_partners_scheduler'" name="function"/>
86+ <field eval="'()'" name="args"/>
87+ </record>
88+
89 <record id="payment_type1" model="base.sale.payment.type">
90- <field name="name">checkmo;cashondelivery</field>
91- <field name="order_policy">manual</field>
92- </record>
93-
94+ <field name="name">checkmo;cashondelivery</field>
95+ <field name="order_policy">manual</field>
96+ </record>
97+
98 <record id="payment_type2" model="base.sale.payment.type">
99- <field name="name">ccsave;free;googlecheckout;paypayl_express;paybox_system;paypal_standard;servired_standard;bbva;cofidis</field>
100- <field name="order_policy">prepaid</field>
101- </record>
102-
103- </data>
104+ <field name="name">ccsave;free;googlecheckout;paypayl_express;paybox_system;paypal_standard;servired_standard;bbva;cofidis</field>
105+ <field name="order_policy">prepaid</field>
106+ </record>
107+
108+
109+ </data>
110 </openerp>
111
112=== modified file 'magentoerpconnect/partner.py'
113--- magentoerpconnect/partner.py 2012-05-16 09:15:39 +0000
114+++ magentoerpconnect/partner.py 2013-01-09 15:14:26 +0000
115@@ -317,4 +317,69 @@
116
117 return False, partner_id
118
119+ def _ext_import_one(self, cr, uid, external_id, vals, external_data, referential_id, defaults=None, context=None):
120+ """ Import one external resource
121+
122+ This method can be inherited to do an action which have to be done
123+ within the same transaction than the import of the resource.
124+ All the database operations done within _ext_import_one
125+ will be rollbacked if any error occurs.
126+ So no action on an external referential should be done within this method.
127+
128+ :param int external_id: id of the resource on the external referential
129+ :param dict vals: vals converted to openerp
130+ :param dict external_data: vals of the external resource before conversion
131+ :param int referential_id: external referential id from where we import the resource
132+ :param dict defaults: defaults value for fields which are not in vals
133+ :return: tuple created id, updated id
134+ """
135+ cid, wid = super(res_partner, self)._ext_import_one(
136+ cr, uid, external_id, vals, external_data, referential_id, defaults=defaults, context=context)
137+
138+ partner_id = cid or wid
139+ if partner_id:
140+ partner = self.browse(cr, uid, partner_id, context=context)
141+
142+ # As a partner without addresses can't be opened
143+ # we create a dummy empty address
144+ if not partner.address:
145+ new_empty_address = [(0, 0, {})]
146+ partner.write({'address': new_empty_address}, context=context)
147+
148+ return cid, wid
149+
150+ def _import_partner_from_external(
151+ self,
152+ cr, uid,
153+ partner_ext_id,
154+ connection,
155+ referential_id,
156+ context=None):
157+ """
158+ Import partner data from magento for a given id
159+
160+ :param partner_ext_id: magento id of customer to import
161+ :param connection: connection data to magento
162+ :param referential_id: referential id from where we import the ressource
163+ """
164+ if context is None:
165+ context = {}
166+ imp_ctx = dict(context,
167+ id=partner_ext_id)
168+ result = self.get_external_data(
169+ cr, uid,
170+ connection,
171+ referential_id,
172+ defaults={},
173+ context=imp_ctx)
174+ partner_id = self.extid_to_oeid(
175+ cr, uid, partner_ext_id, referential_id, context=context)
176+
177+ # update the address book of the customer, this addresses
178+ # will be available in the partner form and the searches
179+ self.import_magento_address_book(
180+ cr, uid, partner_id, referential_id, context=context)
181+
182+ return result
183+
184 res_partner()
185
186=== modified file 'magentoerpconnect/sale.py'
187--- magentoerpconnect/sale.py 2012-12-06 08:08:36 +0000
188+++ magentoerpconnect/sale.py 2013-01-09 15:14:26 +0000
189@@ -30,6 +30,7 @@
190 from tools.translate import _
191 import string
192 #from datetime import datetime
193+from contextlib import closing
194 import tools
195 import time
196 from tools import DEFAULT_SERVER_DATETIME_FORMAT
197@@ -49,6 +50,113 @@
198 'waiting_date': 'holded'}
199 SALE_ORDER_IMPORT_STEP = 200
200
201+
202+class external_shop_group(magerp_osv.magerp_osv):
203+ _inherit = 'external.shop.group'
204+
205+ @staticmethod
206+ def _get_magento_partners(connection, from_date=False, website_id=False):
207+ """
208+ Get data from magento of new and updated customers since a specific_date
209+
210+ :param connection: connection data to magento
211+ :param from_data: date from which we get the changes
212+ """
213+ filters = []
214+ if from_date:
215+ filters = [
216+ {'created_at': {'gt': from_date}}, # OR
217+ {'updated_at': {'gt': from_date}}
218+ ]
219+ if website_id:
220+ if filters:
221+ for f in filters:
222+ f.update(website_id={'eq': website_id})
223+ else:
224+ filters = [{'website_id': {'eq': website_id}}]
225+
226+ data = connection.call('customer.list', filters)
227+ return data
228+
229+ def _import_partners(self, cr, uid, group, context=None):
230+ """
231+ Import partners for a single shop group
232+
233+ :param group: browse record of an external.shop.group
234+ """
235+ if context is None: context = {}
236+
237+ referential_id = group.referential_id.id
238+ from_date = group.import_partners_from_date
239+ website_id = self.oeid_to_extid(cr, uid, group.id, referential_id)
240+ partner_obj = self.pool.get('res.partner')
241+ result = super(external_shop_group, self)._import_partners(
242+ cr, uid, group,
243+ context=context)
244+
245+ ref_obj = self.pool.get('external.referential')
246+ connection = context.get('conn_obj') or \
247+ ref_obj.external_connection(
248+ cr, uid, referential_id, context=context)
249+
250+ [result.setdefault(key, []) for key in ['create_ids', 'write_ids', 'unchanged_ids']]
251+
252+ # Get partners from magento which where created or updated
253+ # since last import
254+ data = self._get_magento_partners(connection, from_date, website_id)
255+
256+ data.sort(key=lambda customer: customer['updated_at'] or customer['created_at'])
257+
258+ imp_ctx = dict(context, import_no_new_cr=True)
259+ with closing(pooler.get_db(cr.dbname).cursor()) as local_cr:
260+ # here we commit each partner updated or created as it can
261+ # be an long processing and we want to be able to restart
262+ # from the state when it failed
263+ for customer in data:
264+ ext_id = customer['customer_id']
265+ change_date = customer['updated_at'] or customer['created_at']
266+ try:
267+ current_result = partner_obj._import_partner_from_external(
268+ local_cr, uid, ext_id,
269+ connection, referential_id,
270+ context=imp_ctx)
271+ except Exception as e:
272+ local_cr.rollback()
273+ if "res_partner_emailid_uniq" in e.message:
274+ message = _("Cannot import a partner: "
275+ "A partner with email %s already exists ")\
276+ % customer['email']
277+ raise osv.except_osv(_("Import Error"), message)
278+ else:
279+ raise
280+ else:
281+ self.write(
282+ local_cr, uid,
283+ group.id,
284+ {'import_partners_from_date': change_date},
285+ context=context)
286+ local_cr.commit()
287+ result['create_ids'] += current_result.get('create_ids')
288+ result['write_ids'] += current_result.get('write_ids')
289+
290+ return result
291+
292+ def run_import_partners_scheduler(self, cr, uid, context=None):
293+ """
294+ Methode to launch the import on all external shop groups
295+ to be used by a cron job
296+ """
297+ group_ids = self.search(cr, uid, [], context=context)
298+
299+ if group_ids:
300+ shop_groups = self.browse(cr, uid, ids, context=context)
301+
302+ for group in shop_groups:
303+ self._import_partners(cr, uid, group, context=context)
304+
305+
306+external_shop_group()
307+
308 class sale_shop(magerp_osv.magerp_osv):
309 _inherit = "sale.shop"
310
311@@ -340,22 +448,39 @@
312 cr, uid, customer_record, referential_id,
313 defaults=partner_defaults, context=context)
314 else:
315+ ext_id = data_record['customer_id']
316+ connection = context.get('conn_obj')
317+ website_id = data_record.get('website_id')
318+ if website_id:
319+ website_id = int(data_record['website_id'])
320+ # check if another customer exists
321+ existing_id = partner_obj.search_magento_partner(
322+ cr, uid,
323+ data_record['customer_email'],
324+ website_id,
325+ context=context)
326+ if existing_id:
327+ # check if the ext_id is the same
328+ # to make sure it isn't some outdated data
329+ # This could happen when a customer change his email
330+ # from A to B on a old account and create
331+ # a new account with A address.
332+ # in such case we want to make sure the old account is
333+ # updated
334+ existing_ext_id = partner_obj.oeid_to_extid(cr, uid, existing_id, referential_id)
335+ if existing_ext_id != ext_id:
336+ partner_obj._import_partner_from_external(
337+ cr, uid, existing_ext_id,
338+ connection, referential_id,
339+ context=context)
340+
341 # always update the customer when importing an order
342- imp_ctx = dict(context)
343- imp_ctx['id'] = data_record['customer_id']
344- partner_obj.get_external_data(
345- cr, uid,
346- context.get('conn_obj'),
347- referential_id,
348- defaults={},
349- context=imp_ctx)
350- partner_id = partner_obj.extid_to_oeid(
351- cr, uid, data_record['customer_id'], referential_id)
352-
353- # update the address book of the customer, this addresses
354- # will be available in the partner form and the searches
355- partner_obj.import_magento_address_book(
356- cr, uid, partner_id, referential_id, context=context)
357+ result = partner_obj._import_partner_from_external(
358+ cr, uid, ext_id,
359+ connection, referential_id,
360+ context=context)
361+ partner_id = (result['create_ids'] and result['create_ids'][0]
362+ or result['write_ids'] and result['write_ids'][0])
363
364 # The addresses of the sale order are imported as active=false
365 # so they are linked with the sale order but they are not displayed