Merge lp:~magentoerpconnect-core-editors/magentoerpconnect/7.0-prototype-multi-worker-notify into lp:magentoerpconnect/7.0-prototype

Proposed by Guewen Baconnier @ Camptocamp
Status: Work in progress
Proposed branch: lp:~magentoerpconnect-core-editors/magentoerpconnect/7.0-prototype-multi-worker-notify
Merge into: lp:magentoerpconnect/7.0-prototype
Prerequisite: lp:~magentoerpconnect-core-editors/magentoerpconnect/7.0-prototype-multi-worker
Diff against target: 60 lines (+34/-0)
2 files modified
connectors/abstract/jobs.py (+1/-0)
connectors/abstract/worker.py (+33/-0)
To merge this branch: bzr merge lp:~magentoerpconnect-core-editors/magentoerpconnect/7.0-prototype-multi-worker-notify
Reviewer Review Type Date Requested Status
MagentoERPConnect core editors Pending
Review via email: mp+147387@code.launchpad.net

This proposal supersedes a proposal from 2013-02-08.

Description of the change

Quick test using Postgresql NOTIFY.

It seems to work at first insight. It could be a way to enqueue jobs in a while the cron would act as a failback for the jobs enqueued in a dead worker.

TODO if we want to use the NOTIFY:
* retrieve the payload from the NOTIFY and assign to worker + enqueue the corresponding job
* add sort of flag on the processes which are allowed to listen the NOTIFY (either the main process in standalone mode, either the cron processes in multiprocess mode).

To post a comment you must log in.

Unmerged revisions

72. By Guewen Baconnier @ Camptocamp

[ADD] test with psql NOTIFY

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'connectors/abstract/jobs.py'
2--- connectors/abstract/jobs.py 2013-02-08 15:10:44 +0000
3+++ connectors/abstract/jobs.py 2013-02-08 15:10:47 +0000
4@@ -122,6 +122,7 @@
5 self.session.uid,
6 vals,
7 self.session.context)
8+ self.session.cr.execute('NOTIFY job, %s', (self.job.id,))
9
10 @property
11 def openerp_id(self):
12
13=== modified file 'connectors/abstract/worker.py'
14--- connectors/abstract/worker.py 2013-02-08 15:10:44 +0000
15+++ connectors/abstract/worker.py 2013-02-08 15:10:47 +0000
16@@ -180,6 +180,35 @@
17 session.uid,
18 context=session.context)
19
20+import select
21+class Listener(threading.Thread):
22+ def __init__(self, db_name, worker):
23+ super(Listener, self).__init__()
24+ self.worker = worker
25+ self.db_name = db_name
26+ self.registry = openerp.pooler.get_pool(db_name)
27+
28+ def run(self):
29+ while 1:
30+ while (self.registry.ready and
31+ 'connectors.installed' in self.registry.models):
32+ self.wait_notify()
33+
34+ time.sleep(WAIT_REGISTRY_TIME)
35+
36+ def wait_notify(self):
37+ db = openerp.sql_db.db_connect(self.db_name)
38+ cr = db.cursor()
39+ cr.autocommit(True)
40+ cr.execute("LISTEN job;")
41+ _logger.debug('Wait Notify')
42+ while 1:
43+ if select.select([cr._cnx], [], [], 5) != ([], [], []):
44+ cr._cnx.poll()
45+ while cr._cnx.notifies:
46+ notify = cr._cnx.notifies.pop()
47+ _logger.debug('Got Notify %s', notify)
48+
49
50 def start_service():
51 # FIXME: registries can change on update of modules or at creation
52@@ -194,4 +223,8 @@
53 watcher.daemon = True
54 watcher.start()
55
56+ listener = Listener(db_name, worker)
57+ listener.daemon = True
58+ listener.start()
59+
60 start_service()

Subscribers

People subscribed via source and target branches