RegistryManager: threading.RLock is missing

Bug #1238560 reported by Guewen Baconnier @ Camptocamp
10
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Odoo Server (MOVED TO GITHUB)
Fix Released
Low
OpenERP Publisher's Warranty Team

Bug Description

Hi,

I just get caught by this bug, which is very hard to reproduce.

It happens only when:
 - you have more than 1 thread which starts when a module is loaded
 - a child thread calls openerp.pooler.get_pool()
 - OpenERP is started from an external script, like "behave" or "oe" when running the python tests (does not seems to happen when started from openerp-server)

It gives errors as this example:

  File ".../server/openerp/service/web_services.py", line 433, in dispatch
    return fn(*params)
  File ".../server/openerp/service/web_services.py", line 438, in exp_login
    res = security.login(db, login, password)
  File ".../server/openerp/service/security.py", line 31, in login
    return user_obj.login(db, login, password)
AttributeError: 'NoneType' object has no attribute 'login'

Cause:

RegistryManager.get() on:
http://bazaar.launchpad.net/~openerp/openobject-server/7.0/view/5096/openerp/modules/registry.py#L185

It must be protected by cls.registries_lock because it may return the registry from cls.registries or create a new one.
What happens when there is no registry and 2 threads calls RegistryManager.get() at the same time:
 1. they both have a KeyError
 2. they both execute RegistryManager.new(). This one is protected by the RLock but that's already too late
 3. the first thread create a new Registry. RegistryManager.get() returns this Registry
 4. then the second thread create a new Registry and replace the first one in cls.registries. RegistryManager.get() returns this Registry
 5. One of the registry (could not see which) has no models => self.pool.get() always returns None

If RegistryManager.get() had been surrounded by a RLock, the second thread would just have waited and would not have created a new Registry.

The branch is coming.

Tags: maintenance

Related branches

description: updated
Changed in openobject-server:
assignee: nobody → OpenERP Publisher's Warranty Team (openerp-opw)
tags: added: maintenance
Changed in openobject-server:
status: New → Confirmed
importance: Undecided → Low
Revision history for this message
Martin Trigaux (OpenERP) (mat-openerp) wrote :

Hello Guewen,

We confirm this could happened and it should be fixed with the lock (we could not reproduce but the analysis seems to confirm it).
We used to have a lock on the get method but it was removed due to deadlock possibilities. This is no longer possible with the current code so we can put the lock back. See my commit message for more information.

revno: 5129 [merge]
revision-id: <email address hidden>

Thanks for the patch

Changed in openobject-server:
status: Confirmed → Fix Released
Revision history for this message
Guewen Baconnier @ Camptocamp (gbaconnier-c2c) wrote :

Thanks for the merge!

I just figured out a way to reproduce it:

Place this snippet in an addons (I put it in server/openerp/addons/base/__init__.py so I'm sure it is loaded).

    import threading
    import openerp

    def get_pool():
        pool = openerp.pooler.get_pool('openerp_database_name')
        pool['ir.module.module']

    def start():
        for __ in range(1, 20):
            thread = threading.Thread(target=get_pool)
            thread.daemon = True
            thread.start()

    for __ in range(1, 4):
        thread = threading.Thread(target=start)
        thread.daemon = True
        thread.start()

When OpenERP starts, it seems that the race condition does never happen.
But it happens when we use "oe":

     oe run-tests -d openerp_database_name -mbase

From time to time, the tests will fail because the threads can't find ir.module.module

no longer affects: ocb-server
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.