Hi
I am working on magentoerpconnect, it's an Openerp module to connect Openerp with magento (e-commerce solution).
In Magento, customer can easly create new fields on a category of product, for example for a camera, he will create a 'number_of_pixel' field.
The first version of magentoerpconnect was able to create dynamically all of this field in openerp (an initial import from magento to openerp was done).
But as customer sell a lot of different products, quickly we face to some limitation.
Some technical limitation, for example postgres allow to create at least 1600 columns per table and performance limitation (reading 1500 useless empty field per product is not so good)
So we decide to store all custom magento field in only one json field.
We choose to override the write, the create and the read function to read/write/create directly the value of a magento field in the json store field called 'magerp'.
https://code.launchpad.net/~akretion-team/+junk/base_json_fields
For example, when I read the field 'x_js_color_magerp', 'x_js_size_magerp' and name, I will call the original read with only the field 'magerp' (which contain all magento value) and 'name' and just after that, the field 'magerp' is "unzipped" to set the correct value in the two fields 'x_js_color_magerp' and 'x_js_size_magerp'.
It's very fast because as I only use call only one read to get the magento field and the openerp field.
(This problem can be also solve with function field, but in this case the read will be call at least 2 times. More other openerp frameword doesn't support the creation of function field dynamically. So I chose the first option.)
It's work great but I have a little problem regarding the creation of the column in openerp.
Indeed for all of this fields, we don't need to create a column as it's "calculate" field.
To problem is that I can not overwrite the function auto_init to cancel the create of some column.
At first I just add an hook on the field orm.py
=== modified file 'openerp/osv/orm.py'
--- openerp/osv/orm.py 2011-07-05 12:22:22 +0000
+++ openerp/osv/orm.py 2011-07-06 23:17:23 +0000
@@ -2714,6 +2714,12 @@
cr.execute('ALTER TABLE "%s" ALTER COLUMN "%s" DROP NOT NULL' % (self._table, column['attname']))
self.__schema.debug("Table '%s': column '%s': dropped NOT NULL constraint",
self._table, column['attname'])
+
+ def _check_if_field_can_be_updated(self, f, context=None):
+ # Don't update custom (also called manual) fields
+ if f.manual and not context.get('update_custom_fields', False):
+ return False
+ return True
def _auto_init(self, cr, context=None):
"""
@@ -2739,7 +2745,6 @@
context = {}
store_compute = False
todo_end = []
- update_custom_fields = context.get('update_custom_fields', False)
self._field_create(cr, context=context)
create = not self._table_exist(cr)
@@ -2767,7 +2772,7 @@
if k in ('id', 'write_uid', 'write_date', 'create_uid', 'create_date'):
continue
# Don't update custom (also called manual) fields
- if f.manual and not update_custom_fields:
+ if not self._check_if_field_can_be_updated(f, context=context):
continue
if isinstance(f, fields.one2many):
There is no big change I have just extracted the check so external module can decide if this field have to be update.
As the function auto_init is very big, maybe it's better to split it and so extract a function which can update only one field.
So I propose this merge.
What do you think?
Thanks
Hi Sebastien,
I had a quick look at your base_json_fields module and it strikes me as quite complicated for what it tries to accomplish. I think the whole principle of having sparse fields stored in a serialization field could be done in a very simple and clean manner, if properly supported by a dedicated column type. This would also avoid having to subclass osv objects and the orm with hacks to enable it.
The change you are proposing seems simple enough indeed, but it makes the osv/business objects sort of 'in control' of what happens to some columns, something that should really be encapsulated in the ORM layer.
The following is not a review of your patch, but a proposal for an alternative approach.
What about creating a new osv.fields type, e.g fields.sparse, that would be a subclass of fields.function? This field would automatically behave as a function field in "multi" mode, making reading and writing several fields on the same osv very cheap. The field would have a mandatory "serialization_
The serialization field could be a normal fields.text, or perhaps we could rewrite/
class sparse(function):
def __init__(self, serialization_
assert serialization_field is proper type
return super(.
def _fnct_write(model, cr, uid, ids, field_name, field_value, fnct_inv_arg, context):
records = model.browse(ids)
for record in records:
# grab serialized value as object - already deserialized
return True
def _fnct_read(model, cr, uid, ids, field_names, arg, context=None):
records = model.browse(ids)
for record in records:
# grab serialized value as object - already deserialized
for field_name in field_names:
return results
Then if we simply make fields.sparse available as a custom field, and show the "serialization_
Storing the magento sparse attributes using fields.sparse would be as simple as adding one fields.serialized column in products, and creating the fields.sparse entries in ir.model.fields on demand. No hacking of osv or orm would be needed at all, as function fields are already properly handled (including during updates).
BTW, the double read for function fields is quite negligible, as it all happens server-side. What generally matters mo...