Merge lp:~openerp-connector-core-editors/openerp-connector/7.0-connector-mapper-refactor into lp:~openerp-connector-core-editors/openerp-connector/7.0

[Guewen Baconnier @ Camptocamp]

Refactoring of the Mappers.
===========================

Work in progress.

[lp:~openerp-connector-core-editors/openerp-connector/7.0-connector-mapper-refactor]

584. By Guewen Baconnier @ Camptocamp on 2013-11-08

[CHG] rename map_record.values(only_create=True) to map_record.values(for_create=True) -> less ambiguous

585. By Guewen Baconnier @ Camptocamp on 2013-11-08

[CHG] move deprecated methods, set some private methods to public (the ones that are really intended to be inherited)

586. By Guewen Baconnier @ Camptocamp on 2013-11-08

[CHG] updated the docstring according to the last developments

587. By Guewen Baconnier @ Camptocamp on 2013-11-08

[FIX] simplify the MapRecord.update() method

588. By Guewen Baconnier @ Camptocamp on 2013-11-08

[IMP] docstring

589. By Guewen Baconnier @ Camptocamp on 2013-11-09

[FIX] mapping options: use an object whose items can be accessed like attributes.

It offers a far better interface as well as for the the call to MapRecord.values() as for the usage in mapping methods

590. By Guewen Baconnier @ Camptocamp on 2013-11-09

[FIX] the backward compatibility code was outdated: binding is the name of the binding to use, to provide only when the relation is not a binding

591. By Guewen Baconnier @ Camptocamp on 2013-11-12

[CHG] MapOptions now returns None when a key is undefined, allowing a better handling in mapping methods when an option is not used

592. By Guewen Baconnier @ Camptocamp on 2013-12-09

[MRG] sync with master branch

593. By Guewen Baconnier @ Camptocamp on 2013-12-09

[MRG] sync with master branch

594. By Guewen Baconnier @ Camptocamp on 2013-12-20

[IMP] extract the build of the MapChild unit instance to allow better reusability

595. By Guewen Baconnier @ Camptocamp on 2013-12-24

[MRG] from lp:openerp-connector

Unmerged revisions

=== modified file 'connector/CHANGES.rst'
--- connector/CHANGES.rst	2013-10-09 18:52:36 +0000
+++ connector/CHANGES.rst	2013-11-08 11:46:46 +0000
@@ -4,6 +4,9 @@
 2.0.1.dev0
 ~~~~~~~~~~
 
+* Add the possibility to use 'Modifiers' functions in the 'direct
+  mappings' (details in the documentation of the Mapper class)
+
 2.0.1 (2013-09-12)
 ~~~~~~~~~~~~~~~~~~
 

=== modified file 'connector/backend.py'
--- connector/backend.py	2013-08-26 20:05:18 +0000
+++ connector/backend.py	2013-11-08 11:46:46 +0000
@@ -20,6 +20,7 @@
 ##############################################################################
 from functools import partial
 from collections import namedtuple
+from .exception import NoConnectorUnitError
 
 __all__ = ['Backend']
 
@@ -257,10 +258,12 @@
         """
         matching_classes = self._get_classes(base_class, session,
                                              model_name)
-        assert matching_classes, ('No matching class found for %s '
-                                  'with session: %s, '
-                                  'model name: %s' %
-                                  (base_class, session, model_name))
+        if not matching_classes:
+            raise NoConnectorUnitError('No matching class found for %s '
+                                       'with session: %s, '
+                                       'model name: %s' %
+                                       (base_class, session, model_name))
+
         assert len(matching_classes) == 1, (
             'Several classes found for %s '
             'with session %s, model name: %s. Found: %s' %

=== modified file 'connector/connector.py'
--- connector/connector.py	2013-09-13 13:22:49 +0000
+++ connector/connector.py	2013-11-08 11:46:46 +0000
@@ -283,11 +283,15 @@
         """
         raise NotImplementedError
 
-    def to_backend(self, binding_id):
+    def to_backend(self, binding_id, wrap=False):
         """ Give the external ID for an OpenERP binding ID
         (ID in a model magento.*)
 
         :param binding_id: OpenERP binding ID for which we want the backend id
+        :param wrap: if False, binding_id is the ID of the binding,
+                     if True, binding_id is the ID of the normal record, the
+                     method will search the corresponding binding and returns
+                     the backend id of the binding
         :return: external ID of the record
         """
         raise NotImplementedError

=== modified file 'connector/doc/guides/concepts.rst'
--- connector/doc/guides/concepts.rst	2013-11-01 09:10:27 +0000
+++ connector/doc/guides/concepts.rst	2013-11-08 11:46:46 +0000
@@ -146,8 +146,7 @@
 systems.
 
 The connector defines some base classes, which you can find below.
-Note that you can define your own ConnectorUnits as well without
-reusing them.
+Note that you can define your own ConnectorUnits as well.
 
 Mappings
 ========
@@ -159,19 +158,16 @@
 
 It supports:
 
-* direct mappings
-
+direct mappings
     Fields *a* is written in field *b*.
 
-* method mappings
-
+method mappings
     A method is used to convert one or many fields to one or many
     fields, with transformation.
     It can be filtered, for example only applied when the record is
     created or when the source fields are modified.
 
-* submapping
-
+submapping
     a sub-record (lines of a sale order) is converted using another
     Mapper
 

=== modified file 'connector/exception.py'
--- connector/exception.py	2013-06-19 14:47:17 +0000
+++ connector/exception.py	2013-11-08 11:46:46 +0000
@@ -24,6 +24,10 @@
     """ Base Exception for the connectors """
 
 
+class NoConnectorUnitError(ConnectorException):
+    """ No ConnectorUnit has been found """
+
+
 class InvalidDataError(ConnectorException):
     """ Data Invalid """
 

=== modified file 'connector/tests/test_backend.py'
--- connector/tests/test_backend.py	2013-08-23 20:34:05 +0000
+++ connector/tests/test_backend.py	2013-11-08 11:46:46 +0000
@@ -6,6 +6,7 @@
 from openerp.addons.connector.backend import (Backend,
                                               get_backend,
                                               BACKENDS)
+from openerp.addons.connector.exception import NoConnectorUnitError
 from openerp.addons.connector.connector import (Binder,
                                                 ConnectorUnit)
 from openerp.addons.connector.unit.mapper import (Mapper,
@@ -73,7 +74,7 @@
                                         self.uid)
 
     def tearDown(self):
-        super(test_backend_register, self).setUp()
+        super(test_backend_register, self).tearDown()
         BACKENDS.backends.clear()
         del self.backend._class_entries[:]
 
@@ -110,7 +111,7 @@
 
     def test_no_register_error(self):
         """ Error when asking for a class and none is found"""
-        with self.assertRaises(AssertionError):
+        with self.assertRaises(NoConnectorUnitError):
             ref = self.backend.get_class(BackendAdapter,
                                          self.session,
                                          'res.users')

=== modified file 'connector/tests/test_mapper.py'
--- connector/tests/test_mapper.py	2013-04-08 11:48:59 +0000
+++ connector/tests/test_mapper.py	2013-11-08 11:46:46 +0000
@@ -2,15 +2,26 @@
 
 import unittest2
 import mock
+import openerp.tests.common as common
 
 from openerp.addons.connector.unit.mapper import (
     Mapper,
     ImportMapper,
+    ImportMapChild,
     MappingDefinition,
     changed_by,
     only_create,
+    convert,
+    m2o_to_backend,
+    backend_to_m2o,
+    none,
     mapping)
 
+from openerp.addons.connector.exception import NoConnectorUnitError
+from openerp.addons.connector.backend import Backend
+from openerp.addons.connector.connector import Environment
+from openerp.addons.connector.session import ConnectorSession
+
 
 class test_mapper(unittest2.TestCase):
     """ Test Mapper """
@@ -151,11 +162,11 @@
         record = {'name': 'Guewen',
                     'street': 'street'}
         mapper = MyMapper(env)
-        mapper.convert(record)
+        map_record = mapper.map_record(record)
         expected = {'out_name': 'Guewen',
                     'out_street': 'STREET'}
-        self.assertEqual(mapper.data, expected)
-        self.assertEqual(mapper.data_for_create, expected)
+        self.assertEqual(map_record.values(), expected)
+        self.assertEqual(map_record.values(only_create=True), expected)
 
     def test_mapping_record_on_create(self):
         """ Map a record and check the result for creation of record """
@@ -176,61 +187,343 @@
         record = {'name': 'Guewen',
                   'street': 'street'}
         mapper = MyMapper(env)
-        mapper.convert(record)
+        map_record = mapper.map_record(record)
         expected = {'out_name': 'Guewen',
                     'out_street': 'STREET'}
-        self.assertEqual(mapper.data, expected)
+        self.assertEqual(map_record.values(), expected)
         expected = {'out_name': 'Guewen',
                     'out_street': 'STREET',
                     'out_city': 'city'}
-        self.assertEqual(mapper.data_for_create, expected)
+        self.assertEqual(map_record.values(only_create=True), expected)
+
+    def test_mapping_update(self):
+        """ Force values on a map record """
+        class MyMapper(ImportMapper):
+
+            direct = [('name', 'out_name')]
+
+            @mapping
+            def street(self, record):
+                return {'out_street': record['street'].upper()}
+
+            @only_create
+            @mapping
+            def city(self, record):
+                return {'out_city': 'city'}
+
+        env = mock.MagicMock()
+        record = {'name': 'Guewen',
+                  'street': 'street'}
+        mapper = MyMapper(env)
+        map_record = mapper.map_record(record)
+        map_record.update(out_city='forced')
+        map_record.update({'test': 1})
+        expected = {'out_name': 'Guewen',
+                    'out_street': 'STREET',
+                    'out_city': 'forced',
+                    'test': 1}
+        self.assertEqual(map_record.values(), expected)
+        expected = {'out_name': 'Guewen',
+                    'out_street': 'STREET',
+                    'out_city': 'forced',
+                    'test': 1}
+        self.assertEqual(map_record.values(only_create=True), expected)
+
+    def test_finalize(self):
+        """ Inherit _finalize to modify values """
+        class MyMapper(ImportMapper):
+
+            direct = [('name', 'out_name')]
+
+            def _finalize(self, record, values):
+                result = super(MyMapper, self)._finalize(record, values)
+                result['test'] = 'abc'
+                return result
+
+        env = mock.MagicMock()
+        record = {'name': 'Guewen',
+                  'street': 'street'}
+        mapper = MyMapper(env)
+        map_record = mapper.map_record(record)
+        expected = {'out_name': 'Guewen',
+                    'test': 'abc'}
+        self.assertEqual(map_record.values(), expected)
+        expected = {'out_name': 'Guewen',
+                    'test': 'abc'}
+        self.assertEqual(map_record.values(only_create=True), expected)
+
+    def test_some_fields(self):
+        """ Map only a selection of fields """
+        class MyMapper(ImportMapper):
+
+            direct = [('name', 'out_name'),
+                      ('street', 'out_street'),
+                      ]
+
+            @changed_by('country')
+            @mapping
+            def country(self, record):
+                return {'country': 'country'}
+
+        env = mock.MagicMock()
+        record = {'name': 'Guewen',
+                  'street': 'street',
+                  'country': 'country'}
+        mapper = MyMapper(env)
+        map_record = mapper.map_record(record)
+        expected = {'out_name': 'Guewen',
+                    'country': 'country'}
+        self.assertEqual(map_record.values(fields=['name', 'country']),
+                         expected)
+        expected = {'out_name': 'Guewen',
+                    'country': 'country'}
+        self.assertEqual(map_record.values(only_create=True,
+                                           fields=['name', 'country']),
+                         expected)
+
+    def test_mapping_modifier(self):
+        """ Map a direct record with a modifier function """
+
+        def do_nothing(field):
+            def transform(self, record, to_attr):
+                return record[field]
+            return transform
+
+        class MyMapper(ImportMapper):
+            direct = [(do_nothing('name'), 'out_name')]
+
+        env = mock.MagicMock()
+        record = {'name': 'Guewen'}
+        mapper = MyMapper(env)
+        map_record = mapper.map_record(record)
+        expected = {'out_name': 'Guewen'}
+        self.assertEqual(map_record.values(), expected)
+        self.assertEqual(map_record.values(only_create=True), expected)
+
+    def test_mapping_convert(self):
+        """ Map a direct record with the convert modifier function """
+        class MyMapper(ImportMapper):
+            direct = [(convert('name', int), 'out_name')]
+
+        env = mock.MagicMock()
+        record = {'name': '300'}
+        mapper = MyMapper(env)
+        map_record = mapper.map_record(record)
+        expected = {'out_name': 300}
+        self.assertEqual(map_record.values(), expected)
+        self.assertEqual(map_record.values(only_create=True), expected)
+
+    def test_mapping_modifier_none(self):
+        """ Pipeline of modifiers """
+        class MyMapper(ImportMapper):
+            direct = [(none('in_f'), 'out_f'),
+                      (none('in_t'), 'out_t')]
+
+        env = mock.MagicMock()
+        record = {'in_f': False, 'in_t': True}
+        mapper = MyMapper(env)
+        map_record = mapper.map_record(record)
+        expected = {'out_f': None, 'out_t': True}
+        self.assertEqual(map_record.values(), expected)
+        self.assertEqual(map_record.values(only_create=True), expected)
+
+    def test_mapping_modifier_pipeline(self):
+        """ Pipeline of modifiers """
+        class MyMapper(ImportMapper):
+            direct = [(none(convert('in_f', bool)), 'out_f'),
+                      (none(convert('in_t', bool)), 'out_t')]
+
+        env = mock.MagicMock()
+        record = {'in_f': 0, 'in_t': 1}
+        mapper = MyMapper(env)
+        map_record = mapper.map_record(record)
+        expected = {'out_f': None, 'out_t': True}
+        self.assertEqual(map_record.values(), expected)
+        self.assertEqual(map_record.values(only_create=True), expected)
+
+    def test_mapping_custom_option(self):
+        """ Usage of custom options in mappings """
+        class MyMapper(ImportMapper):
+            @mapping
+            def any(self, record):
+                if self.options.get('custom'):
+                    res = True
+                else:
+                    res = False
+                return {'res': res}
+
+        env = mock.MagicMock()
+        record = {}
+        mapper = MyMapper(env)
+        map_record = mapper.map_record(record)
+        expected = {'res': True}
+        self.assertEqual(map_record.values(options=dict(custom=True)),
+                         expected)
+
+
+class test_mapper_binding(common.TransactionCase):
+    """ Test Mapper with Bindings"""
+
+    def setUp(self):
+        super(test_mapper_binding, self).setUp()
+        self.session = ConnectorSession(self.cr, self.uid)
+        self.Partner = self.registry('res.partner')
+        self.backend = mock.Mock(wraps=Backend('x', version='y'),
+                                 name='backend')
+        backend_record = mock.Mock()
+        backend_record.get_backend.return_value = self.backend
+        self.env = Environment(backend_record, self.session, 'res.partner')
+        self.country_binder = mock.Mock(name='country_binder')
+        self.country_binder.return_value = self.country_binder
+        self.backend.get_class.return_value = self.country_binder
+
+    def test_mapping_m2o_to_backend(self):
+        """ Map a direct record with the m2o_to_backend modifier function """
+        class MyMapper(ImportMapper):
+            _model_name = 'res.partner'
+            direct = [(m2o_to_backend('country_id'), 'country')]
+
+        partner_id = self.ref('base.main_partner')
+        self.Partner.write(self.cr, self.uid, partner_id,
+                           {'country_id': self.ref('base.ch')})
+        partner = self.Partner.browse(self.cr, self.uid, partner_id)
+        self.country_binder.to_backend.return_value = 10
+
+        mapper = MyMapper(self.env)
+        map_record = mapper.map_record(partner)
+        self.assertEqual(map_record.values(), {'country': 10})
+        self.country_binder.to_backend.assert_called_once_with(
+            partner.country_id.id, wrap=False)
+
+    def test_mapping_backend_to_m2o(self):
+        """ Map a direct record with the backend_to_m2o modifier function """
+        class MyMapper(ImportMapper):
+            _model_name = 'res.partner'
+            direct = [(backend_to_m2o('country'), 'country_id')]
+
+        record = {'country': 10}
+        self.country_binder.to_openerp.return_value = 44
+        mapper = MyMapper(self.env)
+        map_record = mapper.map_record(record)
+        self.assertEqual(map_record.values(), {'country_id': 44})
+        self.country_binder.to_openerp.assert_called_once_with(
+            10, unwrap=False)
+
+    def test_mapping_record_children_no_map_child(self):
+        """ Map a record with children, using default MapChild """
+
+        backend = Backend('backend', '42')
+
+        @backend
+        class LineMapper(ImportMapper):
+            _model_name = 'res.currency.rate'
+            direct = [('name', 'name')]
+
+            @mapping
+            def price(self, record):
+                return {'rate': record['rate'] * 2}
+
+            @only_create
+            @mapping
+            def discount(self, record):
+                return {'test': .5}
+
+        @backend
+        class ObjectMapper(ImportMapper):
+            _model_name = 'res.currency'
+
+            direct = [('name', 'name')]
+
+            children = [('lines', 'line_ids', 'res.currency.rate')]
+
+
+        backend_record = mock.Mock()
+        backend_record.get_backend.side_effect = lambda *a: backend
+        env = Environment(backend_record, self.session, 'res.currency')
+
+        record = {'name': 'SO1',
+                  'lines': [{'name': '2013-11-07',
+                             'rate': 10},
+                            {'name': '2013-11-08',
+                             'rate': 20}]}
+        mapper = ObjectMapper(env)
+        map_record = mapper.map_record(record)
+        expected = {'name': 'SO1',
+                    'line_ids': [(0, 0, {'name': '2013-11-07',
+                                         'rate': 20}),
+                                 (0, 0, {'name': '2013-11-08',
+                                         'rate': 40})]
+                    }
+        self.assertEqual(map_record.values(), expected)
+        expected = {'name': 'SO1',
+                    'line_ids': [(0, 0, {'name': '2013-11-07',
+                                         'rate': 20,
+                                         'test': .5}),
+                                 (0, 0, {'name': '2013-11-08',
+                                         'rate': 40,
+                                         'test': .5})]
+                    }
+        self.assertEqual(map_record.values(only_create=True), expected)
 
     def test_mapping_record_children(self):
-        """ Map a record with children and check the result """
-
+        """ Map a record with children, using defined MapChild """
+
+        backend = Backend('backend', '42')
+
+        @backend
         class LineMapper(ImportMapper):
+            _model_name = 'res.currency.rate'
             direct = [('name', 'name')]
 
             @mapping
             def price(self, record):
-                return {'price': record['price'] * 2}
+                return {'rate': record['rate'] * 2}
 
             @only_create
             @mapping
             def discount(self, record):
-                return {'discount': .5}
-
-        class SaleMapper(ImportMapper):
+                return {'test': .5}
+
+        @backend
+        class SaleLineImportMapChild(ImportMapChild):
+            _model_name = 'res.currency.rate'
+            def _format_items(self, items_values):
+                return [('ABC', values) for values in items_values]
+
+
+        @backend
+        class ObjectMapper(ImportMapper):
+            _model_name = 'res.currency'
 
             direct = [('name', 'name')]
 
-            children = [('lines', 'line_ids', 'sale.order.line')]
-
-            def _init_child_mapper(self, model_name):
-                return LineMapper(self.environment)
-
-        env = mock.MagicMock()
+            children = [('lines', 'line_ids', 'res.currency.rate')]
+
+
+        backend_record = mock.Mock()
+        backend_record.get_backend.side_effect = lambda *a: backend
+        env = Environment(backend_record, self.session, 'res.currency')
 
         record = {'name': 'SO1',
-                  'lines': [{'name': 'Mango',
-                             'price': 10},
-                            {'name': 'Pumpkin',
-                             'price': 20}]}
-        mapper = SaleMapper(env)
-        mapper.convert(record)
-        expected = {'name': 'SO1',
-                    'line_ids': [(0, 0, {'name': 'Mango',
-                                         'price': 20}),
-                                 (0, 0, {'name': 'Pumpkin',
-                                         'price': 40})]
-                    }
-        self.assertEqual(mapper.data, expected)
-        expected = {'name': 'SO1',
-                    'line_ids': [(0, 0, {'name': 'Mango',
-                                         'price': 20,
-                                         'discount': .5}),
-                                 (0, 0, {'name': 'Pumpkin',
-                                         'price': 40,
-                                         'discount': .5})]
-                    }
-        self.assertEqual(mapper.data_for_create, expected)
+                  'lines': [{'name': '2013-11-07',
+                             'rate': 10},
+                            {'name': '2013-11-08',
+                             'rate': 20}]}
+        mapper = ObjectMapper(env)
+        map_record = mapper.map_record(record)
+        expected = {'name': 'SO1',
+                    'line_ids': [('ABC', {'name': '2013-11-07',
+                                          'rate': 20}),
+                                 ('ABC', {'name': '2013-11-08',
+                                          'rate': 40})]
+                    }
+        self.assertEqual(map_record.values(), expected)
+        expected = {'name': 'SO1',
+                    'line_ids': [('ABC', {'name': '2013-11-07',
+                                          'rate': 20,
+                                          'test': .5}),
+                                 ('ABC', {'name': '2013-11-08',
+                                          'rate': 40,
+                                          'test': .5})]
+                    }
+        self.assertEqual(map_record.values(only_create=True), expected)

=== modified file 'connector/unit/mapper.py'
--- connector/unit/mapper.py	2013-10-28 15:17:17 +0000
+++ connector/unit/mapper.py	2013-11-08 11:46:46 +0000
@@ -21,9 +21,10 @@
 
 import logging
 from collections import namedtuple
+from contextlib import contextmanager
 
 from ..connector import ConnectorUnit, MetaConnectorUnit, Environment
-from ..exception import MappingError
+from ..exception import MappingError, NoConnectorUnitError
 
 _logger = logging.getLogger(__name__)
 
@@ -60,6 +61,142 @@
     return func
 
 
+def none(field):
+    """ A modifier intended to be used on the ``direct`` mappings.
+
+    Replace the False-ish values by None.
+    It can be used in a pipeline of modifiers when .
+
+    Example::
+
+        direct = [(none('source'), 'target'),
+                  (none(m2o_to_backend('rel_id'), 'rel_id')]
+
+    :param field: name of the source field in the record
+    :param binding: True if the relation is a binding record
+    """
+    def modifier(self, record, to_attr):
+        if callable(field):
+            result = field(self, record, to_attr)
+        else:
+            result = record[field]
+        if not result:
+            return None
+        return result
+    return modifier
+
+
+def convert(field, conv_type):
+    """ A modifier intended to be used on the ``direct`` mappings.
+
+    Convert a field's value to a given type.
+
+    Example::
+
+        direct = [(convert('source', str), 'target')]
+
+    :param field: name of the source field in the record
+    :param binding: True if the relation is a binding record
+    """
+    def modifier(self, record, to_attr):
+        value = record[field]
+        if not value:
+            return False
+        return conv_type(value)
+    return modifier
+
+
+def m2o_to_backend(field, binding=None):
+    """ A modifier intended to be used on the ``direct`` mappings.
+
+    For a many2one, get the ID on the backend and returns it.
+
+    When the field's relation is not a binding (i.e. it does not point to
+    something like ``magento.*``), the binding model needs to be provided
+    in the ``binding`` keyword argument.
+
+    Example::
+
+        direct = [(m2o_to_backend('country_id', binding='magento.res.country'),
+                   'country'),
+                  (m2o_to_backend('magento_country_id'), 'country')]
+
+    :param field: name of the source field in the record
+    :param binding: name of the binding model is the relation is not a binding
+    """
+    def modifier(self, record, to_attr):
+        if not record[field]:
+            return False
+        column = self.model._all_columns[field].column
+        if column._type != 'many2one':
+            raise ValueError('The column %s should be a many2one, got %s' %
+                             field, column._type)
+        rel_id = record[field].id
+        if binding is None:
+            binding_model = column._obj
+        else:
+            binding_model = binding
+        binder = self.get_binder_for_model(binding_model)
+        # if a relation is not a binding, we wrap the record in the
+        # binding, we'll return the id of the binding
+        wrap = bool(binding)
+        value = binder.to_backend(rel_id, wrap=wrap)
+        if not value:
+            raise MappingError("Can not find an external id for record "
+                               "%s in model %s %s wrapping" %
+                               (rel_id, binding_model,
+                                'with' if wrap else 'without'))
+        return value
+    return modifier
+
+
+def backend_to_m2o(field, binding=None, with_inactive=False):
+    """ A modifier intended to be used on the ``direct`` mappings.
+
+    For a field from a backend which is an ID, search the corresponding
+    binding in OpenERP and returns its ID.
+
+    When the field's relation is not a binding (i.e. it does not point to
+    something like ``magento.*``), the binding model needs to be provided
+    in the ``binding`` keyword argument.
+
+    Example::
+
+        direct = [(backend_to_m2o('country', binding='magento.res.country'),
+                   'country_id'),
+                  (backend_to_m2o('country'), 'magento_country_id')]
+
+    :param field: name of the source field in the record
+    :param binding: name of the binding model is the relation is not a binding
+    :param with_inactive: include the inactive records in OpenERP in the search
+    """
+    def modifier(self, record, to_attr):
+        if not record[field]:
+            return False
+        column = self.model._all_columns[to_attr].column
+        if column._type != 'many2one':
+            raise ValueError('The column %s should be a many2one, got %s' %
+                             to_attr, column._type)
+        rel_id = record[field]
+        if binding is None:
+            binding_model = column._obj
+        else:
+            binding_model = binding
+        binder = self.get_binder_for_model(binding_model)
+        # if we want the ID of a normal record, not a binding,
+        # we ask the unwrapped id to the binder
+        unwrap = bool(binding)
+        with self.session.change_context({'active_test': False}):
+            value = binder.to_openerp(rel_id, unwrap=unwrap)
+        if not value:
+            raise MappingError("Can not find an existing %s for external "
+                               "record %s %s unwrapping" %
+                               (binding_model, rel_id,
+                                'with' if unwrap else 'without'))
+        return value
+    return modifier
+
+
 MappingDefinition = namedtuple('MappingDefinition',
                                ['changed_by',
                                 'only_create'])
@@ -105,8 +242,189 @@
         return cls
 
 
+class MapChild(ConnectorUnit):
+    """
+
+    """
+    _model_name = None
+
+    def _child_mapper(self):
+        raise NotImplementedError
+
+    def _skip_convert_child(self, map_record):
+        """ Hook to implement in sub-classes when some child
+        records should be skipped.
+
+        The parent record is accessible in ``map_record``.
+        If it returns True, the current child record is skipped.
+
+        :param map_record: record that we are converting
+        :type map_record: :py:class:`MapRecord`
+        """
+        return False
+
+    def get_items(self, items, parent, to_attr, options):
+        mapper = self._child_mapper()
+        mapped = []
+        for item in items:
+            map_record = mapper.map_record(item, parent=parent)
+            if self._skip_convert_child(map_record):
+                continue
+            mapped.append(self._get_item_values(map_record, to_attr, options))
+        return self._format_items(mapped)
+
+    def _get_item_values(self, map_record, to_attr, options):
+        """ Get the values from the child Mappers for the items.
+
+        It can be overridden for instance to:
+
+        * Change options
+        * Use a Binder to know if an item already exists to modify an
+          existing item, rather than to add it
+
+        :param map_record: record that we are converting
+        :type map_record: :py:class:`MapRecord`
+        :param to_attr: destination field (can be used for introspecting
+                        the relation)
+        :type to_attr: str
+        :param options: dict of options, herited from the main mapper
+        """
+        return map_record.values(options=options)
+
+    def _format_items(self, items_values):
+        """ Format the values of the items mapped from the child Mappers.
+
+        It can be overridden for instance to add the OpenERP
+        relationships commands ``(6, 0, [IDs])``, ...
+
+        As instance, it can be modified to handle update of existing
+        items: check if 'id' existings in values, use the ``(1, ID,
+        {values}``) command
+
+        :param items_values: list of values for the items to create
+        """
+        return items_values
+
+
+class ImportMapChild(MapChild):
+
+    def _child_mapper(self):
+        return self.get_connector_unit_for_model(ImportMapper, self.model._name)
+
+    def _format_items(self, items_values):
+        """ Format the values of the items mapped from the child Mappers.
+
+        It can be overridden for instance to add the OpenERP
+        relationships commands ``(6, 0, [IDs])``, ...
+
+        As instance, it can be modified to handle update of existing
+        items: check if 'id' existings in values, use the ``(1, ID,
+        {values}``) command
+
+        :param items_values: list of values for the items to create
+        """
+        return [(0, 0, values) for values in items_values]
+
+
+class ExportMapChild(MapChild):
+
+    def _child_mapper(self):
+        return self.get_connector_unit_for_model(ExportMapper, self.model._name)
+
+
 class Mapper(ConnectorUnit):
-    """ Transform a record to a defined output """
+    """ A Mapper translates an external record to an OpenERP record and
+    conversely. The output of a Mapper is a ``dict``.
+
+    3 types of mappings are supported:
+
+    Direct Mappings
+        Example::
+
+            direct = [('source', 'target')]
+
+        Here, the ``source`` field will be copied in the ``target`` field.
+
+        A modifier can be used in the source item.
+        The modifier will be applied to the source field before being
+        copied in the target field.
+        It should be a closure function respecting this idiom::
+
+            def a_function(field):
+                ''' ``field`` is the name of the source field '''
+                def modifier(self, record, to_attr):
+                    ''' self is the current Mapper,
+                        record is the current record to map,
+                        to_attr is the target field'''
+                    return record[field]
+                return modifier
+
+        And used like that::
+
+            direct = [
+                    (a_function('source'), 'target'),
+            ]
+
+        A more concrete example of modifier::
+
+            def convert(field, conv_type):
+                ''' Convert the source field to a defined ``conv_type``
+                (ex. str) before returning it'''
+                def modifier(self, record, to_attr):
+                    value = record[field]
+                    if not value:
+                        return None
+                    return conv_type(value)
+            return modifier
+
+        And used like that::
+
+            direct = [
+                    (convert('myfield', float), 'target_field'),
+            ]
+
+        More examples of modifiers:
+
+        * :py:func:`convert`
+        * :py:func:`m2o_to_backend`
+        * :py:func:`backend_to_m2o`
+
+    Method Mappings
+        A mapping method allows to execute arbitrary code and return one
+        or many fields::
+
+            @mapping
+            def compute_state(self, record):
+                # compute some state, using the ``record`` or not
+                state = 'pending'
+                return {'state': state}
+
+        We can also specify that a mapping methods should be applied
+        only when an object is created, and never applied on further
+        updates::
+
+            @only_create
+            @mapping
+            def default_warehouse(self, record):
+                # get default warehouse
+                warehouse_id = ...
+                return {'warehouse_id': warehouse_id}
+
+    Submappings
+        When a record contains sub-items, like the lines of a sales order,
+        we can convert the children using another Mapper::
+
+            children = [('items', 'line_ids', LineMapper)]
+
+    Usage of a Mapper::
+
+        mapper = Mapper(env)
+        map_record = mapper.map_record()
+        values = map_record.values()
+        values = map_record.values(only_create=True)
+        values = map_record.values(fields=['name', 'street'])
+
+    """
 
     __metaclass__ = MetaMapper
 
@@ -118,6 +436,8 @@
 
     _map_methods = None
 
+    _map_child_class = None
+
     def __init__(self, environment):
         """
 
@@ -125,191 +445,282 @@
         :type environment: :py:class:`connector.connector.Environment`
         """
         super(Mapper, self).__init__(environment)
-        self._data = None
-        self._data_for_create = None
-        self._data_children = None
-
-    def _init_child_mapper(self, model_name):
-        raise NotImplementedError
+        self._options = None
 
     def _after_mapping(self, result):
-        return result
+        """ Mapper.convert_child() has been deprecated """
+        raise DeprecationWarning('Mapper._after_mapping() has been deprecated, '
+                                 'use Mapper._finalize()')
 
     def _map_direct(self, record, from_attr, to_attr):
+        """ Apply the ``direct`` mappings.
+
+        :param record: record to convert from a source to a target
+        :param from_attr: name of the source attribute or a callable
+        :type from_attr: callable | str
+        :param to_attr: name of the target attribute
+        :type to_attr: str
+        """
         raise NotImplementedError
 
     def _map_children(self, record, attr, model):
         raise NotImplementedError
 
+    def skip_convert_child(self, record, parent_values=None):
+        """ Hook to implement in sub-classes when some child
+        records should be skipped.
+
+        This method is only relevable for child mappers.
+
+        If it returns True, the current child record is skipped."""
+        return False
+
     @property
     def map_methods(self):
         for meth, definition in self._map_methods.iteritems():
             yield getattr(self, meth), definition
 
-    def _convert(self, record, fields=None, parent_values=None):
-        if fields is None:
-            fields = {}
-
-        self._data = {}
-        self._data_for_create = {}
-        self._data_children = {}
-
-        _logger.debug('converting record %s to model %s', record, self._model_name)
+    def convert_child(self, record, parent_values=None):
+        """ Mapper.convert_child() has been deprecated """
+        raise DeprecationWarning('Mapper.convert_child() has been deprecated, '
+                                 'use Mapper.map_record() then '
+                                 'map_record.values() ')
+
+    def convert(self, record, fields=None):
+        """ Mapper.convert() has been deprecated
+
+        The usage of a Mapper is now::
+
+            map_record = Mapper(env).map_record(record)
+            values = map_record.values()
+
+        See :py:meth:`Mapper.map_record`, :py:meth:`MapRecord.values`
+
+        """
+        raise DeprecationWarning('Mapper.convert() has been deprecated, '
+                                 'use Mapper.map_record() then '
+                                 'map_record.values() ')
+
+    @property
+    def data(self):
+        """ Mapper.data has been deprecated
+
+        See :py:meth:`Mapper.map_record`, :py:meth:`MapRecord.values`
+
+        """
+        raise DeprecationWarning('Mapper.data has been deprecated, '
+                                 'use Mapper.map_record() then '
+                                 'map_record.values() ')
+
+    @property
+    def data_for_create(self):
+        """ Mapper.data has been deprecated
+
+        See :py:meth:`Mapper.map_record`, :py:meth:`MapRecord.values`
+        """
+        raise DeprecationWarning('Mapper.data_for_create has been deprecated, '
+                                 'use Mapper.map_record() then '
+                                 'map_record.values() ')
+
+    def _map_child(self, map_record, from_attr, to_attr, model_name):
+        assert self._map_child_class is not None, "_map_child_class required"
+        child_records = map_record.source[from_attr]
+        try:
+            mapper_child = self.get_connector_unit_for_model(
+                self._map_child_class, model_name)
+        except NoConnectorUnitError:
+            # does not force developers to use a MapChild ->
+            # will use the default one if not explicitely defined
+            env = Environment(self.backend_record,
+                              self.session,
+                              model_name)
+            mapper_child = self._map_child_class(env)
+        items = mapper_child.get_items(child_records, map_record,
+                                       to_attr, options=self.options)
+        return items
+
+    @contextmanager
+    def _mapping_options(self, options):
+        """ Change the mapping options for the Mapper.
+
+        Context Manager to use in order to alter the behavior
+        of the mapping, when using ``_apply`` or ``finalize``.
+
+        """
+        current = self._options
+        self._options = options
+        yield
+        self._options = current
+
+    @property
+    def options(self):
+        """ Should not be modified manually, only with _mapping_options.
+
+        Options can be accessed in the mapping methods.
+
+        """
+        return self._options
+
+    def map_record(self, record, parent=None):
+        """ Get a MapRecord with record, ready to be converted using the
+        current Mapper.
+
+        :param record: record to transform
+        """
+        return MapRecord(self, record, parent=parent)
+
+    def _apply(self, map_record, options=None):
+        if options is None:
+            options = {}
+        with self._mapping_options(options):
+            return self._apply_with_options(map_record)
+
+    def _apply_with_options(self, map_record):
+        assert self.options is not None, (
+            "options should be defined with '_mapping_options'")
+        _logger.debug('converting record %s to model %s',
+                      map_record.source, self.model._name)
+
+        fields = self.options.get('fields')
+        only_create = self.options.get('only_create')
+        result = {}
         for from_attr, to_attr in self.direct:
             if (not fields or from_attr in fields):
-                # XXX not compatible with all
-                # record type (wrap
-                # records in a standard class representation?)
-                value = self._map_direct(record,
+                value = self._map_direct(map_record.source,
                                          from_attr,
                                          to_attr)
-                self._data[to_attr] = value
+                result[to_attr] = value
 
         for meth, definition in self.map_methods:
             changed_by = definition.changed_by
             if (not fields or not changed_by or
                     changed_by.intersection(fields)):
-                values = meth(record)
+                values = meth(map_record.source)
                 if not values:
                     continue
                 if not isinstance(values, dict):
                     raise ValueError('%s: invalid return value for the '
                                      'mapping method %s' % (values, meth))
-                if definition.only_create:
-                    self._data_for_create.update(values)
-                else:
-                    self._data.update(values)
+                if not definition.only_create or only_create:
+                    result.update(values)
 
         for from_attr, to_attr, model_name in self.children:
             if (not fields or from_attr in fields):
-                self._map_child(record, from_attr, to_attr, model_name)
-
-    def skip_convert_child(self, record, parent_values=None):
-        """ Hook to implement in sub-classes when some child
-        records should be skipped.
-
-        This method is only relevable for child mappers.
-
-        If it returns True, the current child record is skipped."""
-        return False
-
-    def convert_child(self, record, parent_values=None):
-        """ Transform child row contained in a main record, only
-        called from another Mapper.
-
-        :param parent_values: openerp record of the containing object
-            (e.g. sale_order for a sale_order_line)
-        """
-        self._convert(record, parent_values=parent_values)
-
-    def convert(self, record, fields=None):
-        """ Transform an external record to an OpenERP record or the opposite
-
-        Sometimes we want to map values only when we create the records.
-        The mapping methods have to be decorated with ``only_create`` to
-        map values only for creation of records.
-
-        :param record: record to transform
-        :param fields: list of fields to convert, if empty, all fields
-                       are converted
-        """
-        self._convert(record, fields=fields)
-
-    @property
-    def data(self):
-        """ Returns a dict for a record processed by
-        :py:meth:`~_convert` """
-        if self._data is None:
-            raise ValueError('Mapper.convert should be called before '
-                             'accessing the data')
-        result = self._data.copy()
-        for attr, mappers in self._data_children.iteritems():
-            child_data = [mapper.data for mapper in mappers]
-            if child_data:
-                result[attr] = self._format_child_rows(child_data)
-        return self._after_mapping(result)
-
-    @property
-    def data_for_create(self):
-        """ Returns a dict for a record processed by
-        :py:meth:`~_convert` to use only for creation of the record. """
-        if self._data is None:
-            raise ValueError('Mapper.convert should be called before '
-                             'accessing the data')
-        result = self._data.copy()
-        result.update(self._data_for_create)
-        for attr, mappers in self._data_children.iteritems():
-            child_data = [mapper.data_for_create for mapper in mappers]
-            if child_data:
-                result[attr] = self._format_child_rows(child_data)
-        return self._after_mapping(result)
-
-    def _format_child_rows(self, child_records):
-        return child_records
-
-    def _map_child(self, record, from_attr, to_attr, model_name):
-        child_records = record[from_attr]
-        self._data_children[to_attr] = []
-        for child_record in child_records:
-            mapper = self._init_child_mapper(model_name)
-            if mapper.skip_convert_child(child_record, parent_values=record):
-                continue
-            mapper.convert_child(child_record, parent_values=record)
-            self._data_children[to_attr].append(mapper)
+                result[to_attr] = self._map_child(map_record, from_attr,
+                                                  to_attr, model_name)
+
+        return self._finalize(map_record, result)
+
+    def _finalize(self, map_record, values):
+        """ Called at the end of the mapping. Can be used to
+        modify the values before returning them.
+
+        :param map_record: source map_record
+        :type map_record: :py:class:`MapRecord`
+        :param values: mapped values
+        :returns: mapped values
+        :rtype: dict
+        """
+        return values
 
 
 class ImportMapper(Mapper):
     """ Transform a record from a backend to an OpenERP record """
 
+    _map_child_class = ImportMapChild
+
     def _map_direct(self, record, from_attr, to_attr):
+        """ Apply the ``direct`` mappings.
+
+        :param record: record to convert from a source to a target
+        :param from_attr: name of the source attribute or a callable
+        :type from_attr: callable | str
+        :param to_attr: name of the target attribute
+        :type to_attr: str
+        """
+        if callable(from_attr):
+            return from_attr(self, record, to_attr)
+
         value = record.get(from_attr)
         if not value:
             return False
 
+        # may be replaced by the explicit backend_to_m2o
         column = self.model._all_columns[to_attr].column
         if column._type == 'many2one':
-            rel_id = record[from_attr]
-            model_name = column._obj
-            binder = self.get_binder_for_model(model_name)
-            value = binder.to_openerp(rel_id)
-
-            if not value:
-                raise MappingError("Can not find an existing %s for external "
-                                   "record %s" % (model_name, rel_id))
+            mapping = backend_to_m2o(from_attr, binding=True)
+            value = mapping(self, record, to_attr)
         return value
 
-    def _init_child_mapper(self, model_name):
-        env = Environment(self.backend_record,
-                          self.session,
-                          model_name)
-        return env.get_connector_unit(ImportMapper)
-
-    def _format_child_rows(self, child_records):
-        return [(0, 0, data) for data in child_records]
-
 
 class ExportMapper(Mapper):
     """ Transform a record from OpenERP to a backend record """
 
+    _map_child_class = ExportMapChild
+
     def _map_direct(self, record, from_attr, to_attr):
+        """ Apply the ``direct`` mappings.
+
+        :param record: record to convert from a source to a target
+        :param from_attr: name of the source attribute or a callable
+        :type from_attr: callable | str
+        :param to_attr: name of the target attribute
+        :type to_attr: str
+        """
+        if callable(from_attr):
+            return from_attr(self, record, to_attr)
+
         value = record[from_attr]
         if not value:
             return False
 
+        # may be replaced by the explicit m2o_to_backend
         column = self.model._all_columns[from_attr].column
         if column._type == 'many2one':
-            rel_id = record[from_attr].id
-            model_name = column._obj
-            binder = self.get_binder_for_model(model_name)
-            value = binder.to_backend(rel_id)
-
-            if not value:
-                raise MappingError("Can not find an external id for record "
-                                   "%s in model %s" % (rel_id, model_name))
+            mapping = m2o_to_backend(from_attr, binding=True)
+            value = mapping(self, record, to_attr)
         return value
 
-    def _init_child_mapper(self, model_name):
-        env = Environment(self.backend_record,
-                          self.session,
-                          model_name)
-        return env.get_connector_unit(ExportMapper)
+
+class MapRecord(object):
+
+    def __init__(self, mapper, source, parent=None):
+        self._source = source
+        self._mapper = mapper
+        self._parent = parent
+        self._forced_values = {}
+
+    @property
+    def source(self):
+        return self._source
+
+    @property
+    def parent(self):
+        return self._parent
+
+    def values(self, only_create=None, fields=None, options=None):
+        """
+
+        Sometimes we want to map values only for creation the records.
+        The mapping methods have to be decorated with ``only_create`` to
+        map values only for creation of records. Then, we must give the
+        ``only_create=True`` argument to ``values()``.
+
+        """
+        if options is None:
+            options = {}
+        if only_create is not None:
+            options['only_create'] = only_create
+        if fields is not None:
+            options['fields'] = fields
+        values = self._mapper._apply(self, options=options)
+        values.update(self._forced_values)
+        return values
+
+    def update(self, *args, **kwargs):
+        if args:
+            assert len(args) == 1, 'dict expected, got: %s' % args
+            assert isinstance(args[0], dict), 'dict expected, got %s' % args
+            self._forced_values.update(args[0])
+        if kwargs:
+            self._forced_values.update(kwargs)