Merge lp:~marcustomlinson/unity-scopes-api/rebinding_logic into lp:unity-scopes-api/devel

[Marcus Tomlinson]

[lp:~marcustomlinson/unity-scopes-api/rebinding_logic]

296. By Marcus Tomlinson on 2014-04-04

Merged devel

297. By Marcus Tomlinson on 2014-04-04

Fixing tests (can't contruct a ZmqObjectProxy with an invalid runtime pointer)

298. By Marcus Tomlinson on 2014-04-08

On locate() a scope can call registry methods during its start() method. Ensure the registry adapter has enough threads available to handle this.

299. By Marcus Tomlinson on 2014-04-08

Merged devel

300. By Marcus Tomlinson on 2014-04-08

Merged send_state_in_run_scope

301. By Marcus Tomlinson on 2014-04-08

Upped middleware timeout for locate to allow enough time for the locate logic to execute.

302. By Marcus Tomlinson on 2014-04-08

Updated locate test for RegistryI to execute an indirect locate call via rebinding.

303. By Marcus Tomlinson on 2014-04-08

Merged devel

304. By Marcus Tomlinson on 2014-04-09

Fixed ScopeMetadataImpl, ObjectAdapter, and ZmqMiddleware tests

305. By Marcus Tomlinson on 2014-04-09

Added cat_registry_mutex_ in order to avoid pushing a result in parallel to it's category being registered.

306. By Marcus Tomlinson on 2014-04-09

Temporary test hack to see if tests pass in CI

307. By Marcus Tomlinson on 2014-04-09

Added RaiiScopeThread structs to Activation_test and Runtime_test to ensure each test is run in isolation

308. By Marcus Tomlinson on 2014-04-09

Ensure that the state receiver is ready to process requests as soon as the registry is added to the middleware.

309. By Marcus Tomlinson on 2014-04-09

Simplified RaiiScopeThread

310. By Marcus Tomlinson on 2014-04-09

If there are pushes still active when finished() is called, give them a second to complete before destroying the ReplyObject.

311. By Marcus Tomlinson on 2014-04-10

Merged devel

312. By Marcus Tomlinson on 2014-04-10

Fixed synchronization race between client and server copies of the category register in SearchReplyImpl and ResultReplyObject.

313. By Marcus Tomlinson on 2014-04-10

Fixed comment

314. By Marcus Tomlinson on 2014-04-10

If there are pushes still active when finished() is called, give them a second to complete before destroying the ReplyObject.

315. By Marcus Tomlinson on 2014-04-10

Make SearchListenerBase implementations thread safe in tests.

316. By Marcus Tomlinson on 2014-04-10

Make PreviewListenerBase implementations thread safe in tests.

317. By Marcus Tomlinson on 2014-04-10

Wait for active pushes to complete on finished() logic moved to ReplyImpl

318. By Marcus Tomlinson on 2014-04-10

Wait for active pushes to complete on finished() logic moved to ReplyI

319. By Marcus Tomlinson on 2014-04-15

Remove test code

320. By Marcus Tomlinson on 2014-04-15

Removed more test code

321. By Marcus Tomlinson on 2014-04-15

Merged devel and fixed conflicts

322. By Marcus Tomlinson on 2014-04-15

Removed unnecessary lock_guards from push() implementations in tests

323. By Marcus Tomlinson on 2014-04-15

Some more cleaning up

324. By Marcus Tomlinson on 2014-04-15

Check for null pointers when assigning registry_ in the ZmqObject constructor

325. By Marcus Tomlinson on 2014-04-15

Expect pushes to occur in the correct order in the smartscopesproxy search test

326. By Marcus Tomlinson on 2014-04-16

Revert changes made to SearchReplyImpl

327. By Marcus Tomlinson on 2014-04-16

Merged serialize-oneway-invoke and resolved conflicts

328. By Marcus Tomlinson on 2014-04-16

Fixed RegistryI_test according to new async standard

329. By Marcus Tomlinson on 2014-04-16

Fixed Runtime_test

330. By Marcus Tomlinson on 2014-04-16

We can't just close a socket directly and leave it lingering in the connection pool, otherwise we attempt to talk to a closed socket the next time around

331. By Marcus Tomlinson on 2014-04-16

Corrected scope_ids std::array initialization

332. By Marcus Tomlinson on 2014-04-17

Addressed review comments

333. By Marcus Tomlinson on 2014-04-17

Merged serialize-oneway-invoke

334. By Marcus Tomlinson on 2014-04-17

Fix Invocation_test due to new rebinding behavior

335. By Marcus Tomlinson on 2014-04-17

Merged serialize-oneway-invoke

336. By Marcus Tomlinson on 2014-04-17

Merged devel

337. By Marcus Tomlinson on 2014-04-17

Merged serialize-oneway-invoke

338. By Marcus Tomlinson on 2014-04-28

Merged serialize-oneway-invoke and resolved conflicts

339. By Marcus Tomlinson on 2014-04-28

Merged devel

340. By Marcus Tomlinson on 2014-04-28

Merged devel

Unmerged revisions

=== modified file 'include/unity/scopes/internal/zmq_middleware/ZmqMiddleware.h'
--- include/unity/scopes/internal/zmq_middleware/ZmqMiddleware.h	2014-04-02 09:24:02 +0000
+++ include/unity/scopes/internal/zmq_middleware/ZmqMiddleware.h	2014-04-08 08:16:38 +0000
@@ -95,7 +95,8 @@
                            RequestMode mode,
                            int64_t timeout);
 
-    std::shared_ptr<ObjectAdapter> find_adapter(std::string const& name, std::string const& endpoint_dir);
+    std::shared_ptr<ObjectAdapter> find_adapter(std::string const& name, std::string const& endpoint_dir,
+                                                std::string const& category);
 
     ZmqProxy safe_add(std::function<void()>& disconnect_func,
                       std::shared_ptr<ObjectAdapter> const& adapter,

=== modified file 'include/unity/scopes/internal/zmq_middleware/ZmqObjectProxy.h'
--- include/unity/scopes/internal/zmq_middleware/ZmqObjectProxy.h	2014-02-03 02:53:33 +0000
+++ include/unity/scopes/internal/zmq_middleware/ZmqObjectProxy.h	2014-04-08 08:16:38 +0000
@@ -24,6 +24,7 @@
 #include <unity/scopes/internal/zmq_middleware/RequestMode.h>
 #include <unity/scopes/internal/zmq_middleware/ZmqMiddleware.h>
 #include <unity/scopes/internal/zmq_middleware/ZmqObjectProxyFwd.h>
+#include <unity/scopes/internal/zmq_middleware/ZmqRegistryProxyFwd.h>
 #include <unity/scopes/internal/zmq_middleware/ZmqReceiver.h>
 
 #include <capnp/message.h>
@@ -75,6 +76,9 @@
     ZmqReceiver invoke_(capnp::MessageBuilder& out_params, int64_t timeout);
 
 private:
+    ZmqReceiver invoke__(capnp::MessageBuilder& out_params, int64_t timeout);
+
+    ZmqRegistryProxy registry_;
     std::string endpoint_;
     std::string identity_;
     std::string category_;

=== modified file 'scoperegistry/scoperegistry.cpp'
--- scoperegistry/scoperegistry.cpp	2014-04-04 09:40:40 +0000
+++ scoperegistry/scoperegistry.cpp	2014-04-08 08:16:38 +0000
@@ -422,27 +422,6 @@
         // the registry of state changes.
         middleware->add_state_receiver_object("StateReceiver", registry->state_receiver());
 
-        // FIXME, HACK HACK HACK HACK
-        // The middleware should spawn scope processes with lookup() on demand.
-        // Because it currently does not have the plumbing, we start every scope immediately.
-        // When the plumbing appears, remove this.
-        for (auto&& pair : local_scopes)
-        {
-            try
-            {
-                registry->locate(pair.first);
-            }
-            catch (NotFoundException const&)
-            {
-                // We ignore this. If the scope config couldn't be found, add_local_scopes()
-                // has already printed an error message.
-            }
-            catch (std::exception const& e)
-            {
-                error("could not start scope " + pair.first + ": " + e.what());
-            }
-        }
-
         // Drop our shared_ptr to the RegistryObject. This means that the registry object
         // is kept alive only via the shared_ptr held by the middleware. If the middleware
         // shuts down, it clears out the active servant map, which destroys the registry

=== modified file 'src/scopes/internal/RuntimeImpl.cpp'
--- src/scopes/internal/RuntimeImpl.cpp	2014-04-03 14:22:02 +0000
+++ src/scopes/internal/RuntimeImpl.cpp	2014-04-08 08:16:38 +0000
@@ -20,6 +20,7 @@
 #include <unity/scopes/internal/ScopeBaseImpl.h>
 
 #include <unity/scopes/internal/DfltConfig.h>
+#include <unity/scopes/internal/MWStateReceiver.h>
 #include <unity/scopes/internal/RegistryConfig.h>
 #include <unity/scopes/internal/RegistryImpl.h>
 #include <unity/scopes/internal/RuntimeConfig.h>
@@ -201,7 +202,13 @@
 
 void RuntimeImpl::run_scope(ScopeBase *const scope_base, std::string const& scope_ini_file)
 {
-    auto mw = factory()->create(scope_id_, "Zmq", "Zmq.ini");
+    // Retrieve the registry middleware and create a proxy to its state receiver
+    RegistryConfig reg_conf(registry_identity_, registry_configfile_);
+    auto reg_runtime = create(registry_identity_, configfile_);
+    auto reg_mw = reg_runtime->factory()->find(registry_identity_, reg_conf.mw_kind());
+    auto reg_state_receiver = reg_mw->create_state_receiver_proxy("StateReceiver");
+
+    auto mw = factory()->create(scope_id_, reg_conf.mw_kind(), reg_conf.mw_configfile());
 
     {
         // dirname modifies its argument, so we need a copy of scope lib path
@@ -223,7 +230,14 @@
     auto scope = unique_ptr<internal::ScopeObject>(new internal::ScopeObject(this, scope_base));
     auto proxy = mw->add_scope_object(scope_id_, move(scope));
 
+    // Inform the registry that this scope is now ready to process requests
+    reg_state_receiver->push_state(scope_id_, StateReceiverObject::State::ScopeReady);
+
     mw->wait_for_shutdown();
+
+    // Inform the registry that this scope is shutting down
+    reg_state_receiver->push_state(scope_id_, StateReceiverObject::State::ScopeStopping);
+
     run_future.get();
 }
 

=== modified file 'src/scopes/internal/zmq_middleware/ZmqMiddleware.cpp'
--- src/scopes/internal/zmq_middleware/ZmqMiddleware.cpp	2014-04-02 09:24:02 +0000
+++ src/scopes/internal/zmq_middleware/ZmqMiddleware.cpp	2014-04-08 08:16:38 +0000
@@ -63,6 +63,13 @@
 char const* reply_suffix = "-r";  // Appended to server_name_ to create reply adapter name
 char const* state_suffix = "-s";  // Appended to server_name_ to create state adapter name
 
+char const* query_category = "Query";       // query adapter category name
+char const* ctrl_category = "QueryCtrl";    // control adapter category name
+char const* reply_category = "Reply";       // reply adapter category name
+char const* state_category = "State";       // state adapter category name
+char const* scope_category = "Scope";       // scope adapter category name
+char const* registry_category = "Registry"; // registry adapter category name
+
 } // namespace
 
 ZmqMiddleware::ZmqMiddleware(string const& server_name, string const& configfile, RuntimeImpl* runtime)
@@ -113,9 +120,11 @@
         {
             // TODO: get directory from config
             // TODO: get pool size from config
+            // N.B. We absolutely MUST have AT LEAST 2 invoke threads
+            // as rebinding is invoked within 2-way invocations.
             {
                 lock_guard<mutex> lock(data_mutex_);
-                invokers_.reset(new ThreadPool(1));
+                invokers_.reset(new ThreadPool(2));
             }
             state_ = Started;
             state_changed_.notify_all();
@@ -273,7 +282,7 @@
     }
 
     // Now run over the map and check each value
-    string category = "Scope";
+    string category = scope_category;
     RequestMode mode = RequestMode::Twoway;
     int64_t timeout = -1;
     for (auto const& pair : fmap)
@@ -343,7 +352,7 @@
     MWRegistryProxy proxy;
     try
     {
-        proxy.reset(new ZmqRegistry(this, endpoint, identity, "Registry", twoway_timeout_));
+        proxy.reset(new ZmqRegistry(this, endpoint, identity, registry_category, twoway_timeout_));
     }
     catch (zmqpp::exception const& e)
     {
@@ -358,7 +367,7 @@
     try
     {
         string endpoint = "ipc://" + config_.private_dir() + "/" + identity;
-        proxy.reset(new ZmqScope(this, endpoint, identity, "Scope", twoway_timeout_));
+        proxy.reset(new ZmqScope(this, endpoint, identity, scope_category, twoway_timeout_));
     }
     catch (zmqpp::exception const& e)
     {
@@ -372,7 +381,7 @@
     MWScopeProxy proxy;
     try
     {
-        proxy.reset(new ZmqScope(this, endpoint, identity, "Scope", twoway_timeout_));
+        proxy.reset(new ZmqScope(this, endpoint, identity, scope_category, twoway_timeout_));
     }
     catch (zmqpp::exception const& e)
     {
@@ -386,7 +395,7 @@
     MWQueryProxy proxy;
     try
     {
-        proxy.reset(new ZmqQuery(this, endpoint, identity, "Query"));
+        proxy.reset(new ZmqQuery(this, endpoint, identity, query_category));
     }
     catch (zmqpp::exception const& e)
     {
@@ -400,7 +409,7 @@
     MWQueryCtrlProxy proxy;
     try
     {
-        proxy.reset(new ZmqQueryCtrl(this, endpoint, identity, "QueryCtrl"));
+        proxy.reset(new ZmqQueryCtrl(this, endpoint, identity, ctrl_category));
     }
     catch (zmqpp::exception const& e)
     {
@@ -415,7 +424,7 @@
     try
     {
         proxy.reset(new ZmqStateReceiver(this, "ipc://" + config_.private_dir() + "/" +  server_name_ + state_suffix,
-                                         identity, "StateReceiver"));
+                                         identity, state_category));
     }
     catch (zmqpp::exception const& e)
     {
@@ -432,11 +441,11 @@
     try
     {
         shared_ptr<QueryCtrlI> qci(make_shared<QueryCtrlI>(ctrl));
-        auto adapter = find_adapter(server_name_ + ctrl_suffix, config_.private_dir());
+        auto adapter = find_adapter(server_name_ + ctrl_suffix, config_.private_dir(), ctrl_category);
         function<void()> df;
         auto proxy = safe_add(df, adapter, "", qci);
         ctrl->set_disconnect_function(df);
-        return ZmqQueryCtrlProxy(new ZmqQueryCtrl(this, proxy->endpoint(), proxy->identity(), "QueryCtrl"));
+        return ZmqQueryCtrlProxy(new ZmqQueryCtrl(this, proxy->endpoint(), proxy->identity(), ctrl_category));
     }
     catch (std::exception const& e) // Should never happen unless our implementation is broken
     {
@@ -454,8 +463,8 @@
     try
     {
         shared_ptr<QueryCtrlI> qci(make_shared<QueryCtrlI>(ctrl));
-        auto adapter = find_adapter(server_name_ + ctrl_suffix, config_.private_dir());
-        auto df = safe_dflt_add(adapter, "QueryCtrl", qci);
+        auto adapter = find_adapter(server_name_ + ctrl_suffix, config_.private_dir(), ctrl_category);
+        auto df = safe_dflt_add(adapter, ctrl_category, qci);
         ctrl->set_disconnect_function(df);
     }
     catch (std::exception const& e) // Should never happen unless our implementation is broken
@@ -474,11 +483,11 @@
     try
     {
         shared_ptr<QueryI> qi(make_shared<QueryI>(query));
-        auto adapter = find_adapter(server_name_ + query_suffix, config_.private_dir());
+        auto adapter = find_adapter(server_name_ + query_suffix, config_.private_dir(), query_category);
         function<void()> df;
         auto proxy = safe_add(df, adapter, "", qi);
         query->set_disconnect_function(df);
-        return ZmqQueryProxy(new ZmqQuery(this, proxy->endpoint(), proxy->identity(), "Query"));
+        return ZmqQueryProxy(new ZmqQuery(this, proxy->endpoint(), proxy->identity(), query_category));
     }
     catch (std::exception const& e) // Should never happen unless our implementation is broken
     {
@@ -496,8 +505,8 @@
     try
     {
         shared_ptr<QueryI> qi(make_shared<QueryI>(query));
-        auto adapter = find_adapter(server_name_ + query_suffix, config_.private_dir());
-        auto df = safe_dflt_add(adapter, "Query", qi);
+        auto adapter = find_adapter(server_name_ + query_suffix, config_.private_dir(), query_category);
+        auto df = safe_dflt_add(adapter, query_category, qi);
         query->set_disconnect_function(df);
     }
     catch (std::exception const& e) // Should never happen unless our implementation is broken
@@ -517,11 +526,11 @@
     try
     {
         shared_ptr<RegistryI> ri(make_shared<RegistryI>(registry));
-        auto adapter = find_adapter(server_name_, runtime()->registry_endpointdir());
+        auto adapter = find_adapter(server_name_, runtime()->registry_endpointdir(), registry_category);
         function<void()> df;
         auto proxy = safe_add(df, adapter, identity, ri);
         registry->set_disconnect_function(df);
-        return ZmqRegistryProxy(new ZmqRegistry(this, proxy->endpoint(), proxy->identity(), "Registry", twoway_timeout_));
+        return ZmqRegistryProxy(new ZmqRegistry(this, proxy->endpoint(), proxy->identity(), registry_category, twoway_timeout_));
     }
     catch (std::exception const& e) // Should never happen unless our implementation is broken
     {
@@ -540,11 +549,11 @@
     try
     {
         shared_ptr<ReplyI> ri(make_shared<ReplyI>(reply));
-        auto adapter = find_adapter(server_name_ + reply_suffix, config_.public_dir());
+        auto adapter = find_adapter(server_name_ + reply_suffix, config_.public_dir(), reply_category);
         function<void()> df;
         auto proxy = safe_add(df, adapter, "", ri);
         reply->set_disconnect_function(df);
-        return ZmqReplyProxy(new ZmqReply(this, proxy->endpoint(), proxy->identity(), "Reply"));
+        return ZmqReplyProxy(new ZmqReply(this, proxy->endpoint(), proxy->identity(), reply_category));
     }
     catch (std::exception const& e) // Should never happen unless our implementation is broken
     {
@@ -564,11 +573,11 @@
     try
     {
         shared_ptr<ScopeI> si(make_shared<ScopeI>(scope));
-        auto adapter = find_adapter(server_name_, config_.private_dir());
+        auto adapter = find_adapter(server_name_, config_.private_dir(), scope_category);
         function<void()> df;
         auto proxy = safe_add(df, adapter, identity, si);
         scope->set_disconnect_function(df);
-        return ZmqScopeProxy(new ZmqScope(this, proxy->endpoint(), proxy->identity(), "Scope", twoway_timeout_));
+        return ZmqScopeProxy(new ZmqScope(this, proxy->endpoint(), proxy->identity(), scope_category, twoway_timeout_));
     }
     catch (std::exception const& e) // Should never happen unless our implementation is broken
     {
@@ -586,8 +595,8 @@
     try
     {
         shared_ptr<ScopeI> si(make_shared<ScopeI>(scope));
-        auto adapter = find_adapter(server_name_, config_.private_dir());
-        auto df = safe_dflt_add(adapter, "Scope", si);
+        auto adapter = find_adapter(server_name_, config_.private_dir(), scope_category);
+        auto df = safe_dflt_add(adapter, scope_category, si);
         scope->set_disconnect_function(df);
     }
     catch (std::exception const& e) // Should never happen unless our implementation is broken
@@ -607,11 +616,11 @@
     try
     {
         shared_ptr<StateReceiverI> sri(make_shared<StateReceiverI>(state_receiver));
-        auto adapter = find_adapter(server_name_ + state_suffix, config_.private_dir());
+        auto adapter = find_adapter(server_name_ + state_suffix, config_.private_dir(), state_category);
         function<void()> df;
         auto proxy = safe_add(df, adapter, identity, sri);
         state_receiver->set_disconnect_function(df);
-        return ZmqStateReceiverProxy(new ZmqStateReceiver(this, proxy->endpoint(), proxy->identity(), "StateReceiver"));
+        return ZmqStateReceiverProxy(new ZmqStateReceiver(this, proxy->endpoint(), proxy->identity(), state_category));
     }
     catch (std::exception const& e) // Should never happen unless our implementation is broken
     {
@@ -675,12 +684,12 @@
     {
         throw MiddlewareException("make_typed_proxy(): cannot create oneway proxies");
     }
-    if (category == "Scope")
+    if (category == scope_category)
     {
         auto p = make_shared<ZmqScope>(this, endpoint, identity, category, timeout);
         return ScopeImpl::create(p, runtime(), identity);
     }
-    else if (category == "Registry")
+    else if (category == registry_category)
     {
         auto p = make_shared<ZmqRegistry>(this, endpoint, identity, category, timeout);
         return make_shared<RegistryImpl>(p, runtime());
@@ -691,23 +700,8 @@
     }
 }
 
-namespace
-{
-
-bool has_suffix(string const& s, string const& suffix)
-{
-    auto s_len = s.length();
-    auto suffix_len = suffix.length();
-    if (s_len >= suffix_len)
-    {
-        return s.compare(s_len - suffix_len, suffix_len, suffix) == 0;
-    }
-    return false;
-}
-
-} // namespace
-
-shared_ptr<ObjectAdapter> ZmqMiddleware::find_adapter(string const& name, string const& endpoint_dir)
+shared_ptr<ObjectAdapter> ZmqMiddleware::find_adapter(string const& name, string const& endpoint_dir,
+                                                      string const& category)
 {
     lock_guard<mutex> lock(data_mutex_);
 
@@ -720,33 +714,49 @@
     // We don't have the requested adapter yet, so we create it on the fly.
     int pool_size;
     RequestMode mode;
-    if (has_suffix(name, query_suffix))
+    if (category == query_category)
     {
         // The query adapter is single or multi-threaded and supports oneway operations only.
         // TODO: get pool size from config
         pool_size = 1;
         mode = RequestMode::Oneway;
     }
-    else if (has_suffix(name, ctrl_suffix))
+    else if (category == ctrl_category)
     {
         // The ctrl adapter is single-threaded and supports oneway operations only.
         pool_size = 1;
         mode = RequestMode::Oneway;
     }
-    else if (has_suffix(name, reply_suffix))
+    else if (category == reply_category)
     {
         // The reply adapter is single- or multi-threaded and supports oneway operations only.
         // TODO: get pool size from config
         pool_size = 1;
         mode = RequestMode::Oneway;
     }
-    else if (has_suffix(name, state_suffix))
+    else if (category == state_category)
     {
         // The state adapter is single- or multi-threaded and supports oneway operations only.
         // TODO: get pool size from config
         pool_size = 1;
         mode = RequestMode::Oneway;
     }
+    else if (category == scope_category)
+    {
+        // The scope adapter is single- or multi-threaded and supports twoway operations only.
+        // TODO: get pool size from config
+        pool_size = 1;
+        mode = RequestMode::Twoway;
+    }
+    else if (category == registry_category)
+    {
+        // The registry adapter is single- or multi-threaded and supports twoway operations only.
+        // TODO: get pool size from config
+        // NB: On rebind, locate() is called on this adapter. A scope may then call registry methods during
+        // its start() method, hence we must ensure this adapter has enough threads available to handle this.
+        pool_size = 6;
+        mode = RequestMode::Twoway;
+    }
     else
     {
         // The normal adapter is single- or multi-threaded and supports twoway operations only.
@@ -757,7 +767,7 @@
 
     // The query adapter is always inproc.
     string endpoint;
-    if (has_suffix(name, query_suffix))
+    if (category == query_category)
     {
         endpoint = "inproc://" + name;
     }

=== modified file 'src/scopes/internal/zmq_middleware/ZmqObject.cpp'
--- src/scopes/internal/zmq_middleware/ZmqObject.cpp	2014-02-03 07:28:36 +0000
+++ src/scopes/internal/zmq_middleware/ZmqObject.cpp	2014-04-08 08:16:38 +0000
@@ -18,9 +18,11 @@
 
 #include <unity/scopes/internal/zmq_middleware/ZmqObjectProxy.h>
 
+#include <unity/scopes/internal/RuntimeImpl.h>
 #include <unity/scopes/internal/zmq_middleware/ConnectionPool.h>
 #include <unity/scopes/internal/zmq_middleware/Util.h>
 #include <unity/scopes/internal/zmq_middleware/ZmqException.h>
+#include <unity/scopes/internal/zmq_middleware/ZmqRegistry.h>
 #include <unity/scopes/internal/zmq_middleware/ZmqSender.h>
 #include <unity/scopes/ScopeExceptions.h>
 
@@ -61,6 +63,14 @@
     assert(timeout >= -1);
     throw_if_bad_endpoint(endpoint);
 
+    // retrieve the registry proxy if this object is not the registry itself
+    auto runtime = mw_base->runtime();
+    if (identity != runtime->registry_identity())
+    {
+        registry_ = dynamic_pointer_cast<ZmqRegistry>(mw_base->create_registry_proxy(
+                                                          runtime->registry_identity(), runtime->registry_endpoint()));
+    }
+
     // Make sure that fields have consistent settings for null proxies.
     if (endpoint.empty() || identity.empty())
     {
@@ -165,11 +175,32 @@
     return invoke_(out_params, timeout_);
 }
 
+ZmqReceiver ZmqObjectProxy::invoke_(capnp::MessageBuilder& out_params, int64_t timeout)
+{
+    try
+    {
+        return invoke__(out_params, timeout);
+    }
+    catch (TimeoutException const&)
+    {
+        if (!registry_)
+        {
+            throw;
+        }
+        ObjectProxy new_proxy = registry_->locate(identity_);
+        endpoint_ = new_proxy->endpoint();
+        identity_ = new_proxy->identity();
+        category_ = new_proxy->category();
+        timeout_ = new_proxy->timeout();
+        return invoke__(out_params, timeout);
+    }
+}
+
 // Get a socket to the endpoint for this proxy and write the request on the wire.
 // For a twoway request, poll for the reply with the timeout set for this proxy.
 // Return a receiver for the response (whether this is a oneway or twoway request).
 
-ZmqReceiver ZmqObjectProxy::invoke_(capnp::MessageBuilder& out_params, int64_t timeout)
+ZmqReceiver ZmqObjectProxy::invoke__(capnp::MessageBuilder& out_params, int64_t timeout)
 {
     // Each calling thread gets its own pool because zmq sockets are not thread-safe.
     thread_local static ConnectionPool pool(*mw_base()->context());

=== modified file 'test/gtest/scopes/Runtime/Runtime_test.cpp'
--- test/gtest/scopes/Runtime/Runtime_test.cpp	2014-04-03 14:22:02 +0000
+++ test/gtest/scopes/Runtime/Runtime_test.cpp	2014-04-08 08:16:38 +0000
@@ -240,7 +240,7 @@
     receiver->wait_until_finished();
 
     auto result = receiver->last_result();
-    EXPECT_TRUE(result.get() != nullptr);
+    ASSERT_TRUE(result.get() != nullptr);
 
     auto target = result->target_scope_proxy();
     EXPECT_TRUE(target != nullptr);

=== modified file 'test/gtest/scopes/internal/ScopeMetadataImpl/ScopeMetadataImpl_test.cpp'
--- test/gtest/scopes/internal/ScopeMetadataImpl/ScopeMetadataImpl_test.cpp	2014-04-04 09:40:40 +0000
+++ test/gtest/scopes/internal/ScopeMetadataImpl/ScopeMetadataImpl_test.cpp	2014-04-08 08:16:38 +0000
@@ -18,6 +18,7 @@
 
 #include <unity/scopes/internal/ScopeMetadataImpl.h>
 
+#include <unity/scopes/internal/RuntimeImpl.h>
 #include <unity/scopes/internal/ScopeImpl.h>
 #include <unity/scopes/internal/zmq_middleware/ZmqMiddleware.h>
 #include <unity/scopes/ScopeExceptions.h>
@@ -35,8 +36,9 @@
 
 TEST(ScopeMetadataImpl, basic)
 {
+    auto rt = RuntimeImpl::create("testscope");
     ZmqMiddleware mw("testscope", TEST_BUILD_ROOT "/gtest/scopes/internal/ScopeMetadataImpl/Zmq.ini",
-                     (RuntimeImpl*)0x1);
+                     rt.get());
 
     unique_ptr<ScopeMetadataImpl> mi(new ScopeMetadataImpl(&mw));
     mi->set_scope_id("scope_id");
@@ -229,8 +231,9 @@
 
 TEST(ScopeMetadataImpl, serialize)
 {
+    auto rt = RuntimeImpl::create("testscope");
     ZmqMiddleware mw("testscope", TEST_BUILD_ROOT "/gtest/scopes/internal/ScopeMetadataImpl/Zmq.ini",
-                     (RuntimeImpl*)0x1);
+                     rt.get());
 
     unique_ptr<ScopeMetadataImpl> mi(new ScopeMetadataImpl(&mw));
     mi->set_scope_id("scope_id");
@@ -281,8 +284,9 @@
 
 TEST(ScopeMetadataImpl, serialize_exceptions)
 {
+    auto rt = RuntimeImpl::create("testscope");
     ZmqMiddleware mw("testscope", TEST_BUILD_ROOT "/gtest/scopes/internal/ScopeMetadataImpl/Zmq.ini",
-                     (RuntimeImpl*)0x1);
+                     rt.get());
 
     ScopeMetadataImpl mi(&mw);
     try
@@ -348,8 +352,9 @@
 
 TEST(ScopeMetadataImpl, deserialize_exceptions)
 {
+    auto rt = RuntimeImpl::create("testscope");
     ZmqMiddleware mw("testscope", TEST_BUILD_ROOT "/gtest/scopes/internal/ScopeMetadataImpl/Zmq.ini",
-                     (RuntimeImpl*)0x1);
+                     rt.get());
 
     VariantMap m;
     try

=== modified file 'test/gtest/scopes/internal/zmq_middleware/ObjectAdapter/ObjectAdapter_test.cpp'
--- test/gtest/scopes/internal/zmq_middleware/ObjectAdapter/ObjectAdapter_test.cpp	2014-01-24 11:16:14 +0000
+++ test/gtest/scopes/internal/zmq_middleware/ObjectAdapter/ObjectAdapter_test.cpp	2014-04-08 08:16:38 +0000
@@ -19,6 +19,7 @@
 #include <unity/scopes/internal/zmq_middleware/ObjectAdapter.h>
 
 #include <scopes/internal/zmq_middleware/capnproto/Message.capnp.h>
+#include <unity/scopes/internal/RuntimeImpl.h>
 #include <unity/scopes/internal/zmq_middleware/ServantBase.h>
 #include <unity/scopes/internal/zmq_middleware/ZmqException.h>
 #include <unity/scopes/internal/zmq_middleware/ZmqMiddleware.h>
@@ -281,8 +282,9 @@
 
 TEST(ObjectAdapter, add_remove_find)
 {
+    auto rt = RuntimeImpl::create("testscope");
     ZmqMiddleware mw("testscope", TEST_BUILD_ROOT "/gtest/scopes/internal/zmq_middleware/ObjectAdapter/Zmq.ini",
-                     (RuntimeImpl*)0x1);
+                     rt.get());
 
     wait();
     ObjectAdapter a(mw, "testscope", "ipc://testscope", RequestMode::Twoway, 5);
@@ -568,8 +570,9 @@
 
 TEST(ObjectAdapter, invoke_ok)
 {
+    auto rt = RuntimeImpl::create("testscope");
     ZmqMiddleware mw("testscope", TEST_BUILD_ROOT "/gtest/scopes/internal/zmq_middleware/ObjectAdapter/Zmq.ini",
-                     (RuntimeImpl*)0x1);
+                     rt.get());
 
     wait();
     ObjectAdapter a(mw, "testscope", "ipc://testscope", RequestMode::Twoway, 1);
@@ -620,8 +623,9 @@
 
 TEST(ObjectAdapter, invoke_object_not_exist)
 {
+    auto rt = RuntimeImpl::create("testscope");
     ZmqMiddleware mw("testscope", TEST_BUILD_ROOT "/gtest/scopes/internal/zmq_middleware/ObjectAdapter/Zmq.ini",
-                     (RuntimeImpl*)0x1);
+                     rt.get());
 
     wait();
     ObjectAdapter a(mw, "testscope", "ipc://testscope", RequestMode::Twoway, 1);
@@ -665,8 +669,9 @@
 
 TEST(ObjectAdapter, invoke_operation_not_exist)
 {
+    auto rt = RuntimeImpl::create("testscope");
     ZmqMiddleware mw("testscope", TEST_BUILD_ROOT "/gtest/scopes/internal/zmq_middleware/ObjectAdapter/Zmq.ini",
-                     (RuntimeImpl*)0x1);
+                     rt.get());
 
     wait();
     ObjectAdapter a(mw, "testscope", "ipc://testscope", RequestMode::Twoway, 1);
@@ -785,8 +790,9 @@
 
 TEST(ObjectAdapter, twoway_threading)
 {
+    auto rt = RuntimeImpl::create("testscope");
     ZmqMiddleware mw("testscope", TEST_BUILD_ROOT "/gtest/scopes/internal/zmq_middleware/ObjectAdapter/Zmq.ini",
-                     (RuntimeImpl*)0x1);
+                     rt.get());
 
     // Single servant to which we send requests concurrently.
     shared_ptr<CountingServant> o(new CountingServant(100));
@@ -819,8 +825,9 @@
 
 TEST(ObjectAdapter, oneway_threading)
 {
+    auto rt = RuntimeImpl::create("testscope");
     ZmqMiddleware mw("testscope", TEST_BUILD_ROOT "/gtest/scopes/internal/zmq_middleware/ObjectAdapter/Zmq.ini",
-                     (RuntimeImpl*)0x1);
+                     rt.get());
 
     // Single servant to which we send requests concurrently.
     shared_ptr<CountingServant> o(new CountingServant(100));
@@ -890,8 +897,9 @@
 
 TEST(ObjectAdapter, servant_map_destructor)
 {
+    auto rt = RuntimeImpl::create("testscope");
     ZmqMiddleware mw("testscope", TEST_BUILD_ROOT "/gtest/scopes/internal/zmq_middleware/ObjectAdapter/Zmq.ini",
-                     (RuntimeImpl*)0x1);
+                     rt.get());
 
     {
         wait();

=== modified file 'test/gtest/scopes/internal/zmq_middleware/ZmqMiddleware/ZmqMiddleware_test.cpp'
--- test/gtest/scopes/internal/zmq_middleware/ZmqMiddleware/ZmqMiddleware_test.cpp	2014-03-07 01:07:28 +0000
+++ test/gtest/scopes/internal/zmq_middleware/ZmqMiddleware/ZmqMiddleware_test.cpp	2014-04-08 08:16:38 +0000
@@ -18,6 +18,7 @@
 
 #include <unity/scopes/internal/zmq_middleware/ZmqMiddleware.h>
 
+#include <unity/scopes/internal/RuntimeImpl.h>
 #include <unity/scopes/internal/MWObjectProxy.h>
 #include <unity/scopes/internal/zmq_middleware/ZmqObjectProxy.h>
 #include <unity/scopes/ScopeExceptions.h>
@@ -44,9 +45,10 @@
 
 TEST(ZmqMiddleware, string_to_proxy)
 {
+    auto rt = RuntimeImpl::create("testscope");
     ZmqMiddleware mw("testscope",
                      TEST_BUILD_ROOT "/gtest/scopes/internal/zmq_middleware/ZmqMiddleware/Zmq.ini",
-                     (RuntimeImpl*)0x1);
+                     rt.get());
 
     ObjectProxy p;
     ScopeProxy sp;