Merge lp:~jk0/nova/lp697019 into lp:~hudson-openstack/nova/trunk

[Josh Kearney]

Recover from a lost data store connection.

[Ed Leafe (ed-leafe)]

The docstring to register_models() in models.py states that it should never have to be called directly, yet here it's obvious that it must. You should update that docstring to explain the case for when a connection is dropped.

Otherwise, it looks good.

review: Approve

[lp:~jk0/nova/lp697019]

517. By Josh Kearney on 2011-01-05

Updated register_models() docstring

[Cory Wright (corywright)]

test and worked as advertised. lgtm.

review: Approve

=== modified file 'nova/db/sqlalchemy/models.py'
--- nova/db/sqlalchemy/models.py	2011-01-03 19:29:39 +0000
+++ nova/db/sqlalchemy/models.py	2011-01-05 22:06:18 +0000
@@ -545,7 +545,8 @@
     """Register Models and create metadata.
 
     Called from nova.db.sqlalchemy.__init__ as part of loading the driver,
-    it will never need to be called explicitly elsewhere.
+    it will never need to be called explicitly elsewhere unless the
+    connection is lost and needs to be reestablished.
     """
     from sqlalchemy import create_engine
     models = (Service, Instance, InstanceActions,

=== modified file 'nova/service.py'
--- nova/service.py	2010-12-22 20:59:53 +0000
+++ nova/service.py	2011-01-05 22:06:18 +0000
@@ -24,17 +24,21 @@
 import logging
 import os
 import sys
+import time
 
 from eventlet import event
 from eventlet import greenthread
 from eventlet import greenpool
 
+from sqlalchemy.exc import OperationalError
+
 from nova import context
 from nova import db
 from nova import exception
 from nova import flags
 from nova import rpc
 from nova import utils
+from nova.db.sqlalchemy import models
 
 
 FLAGS = flags.FLAGS
@@ -204,6 +208,14 @@
                 self.model_disconnected = True
                 logging.exception(_("model server went away"))
 
+                try:
+                    models.register_models()
+                except OperationalError:
+                    logging.exception(_("Data store is unreachable."
+                        " Trying again in %d seconds.") %
+                            FLAGS.sql_retry_interval)
+                    time.sleep(FLAGS.sql_retry_interval)
+
 
 def serve(*services):
     argv = FLAGS(sys.argv)