Merge lp:~smspillaz/nux/nux.fix_1097281 into lp:nux

[Sam Spilsbury]

Implement a message-passing system in nux::WindowThread and nux::ProgramFramework.

This makes it possible to define and implement protocols for WindowThreads to
communicate with each other in a thread-safe way without having to resort to
using window system events or the like.

New file descriptors can be added for watching with a watch-callback executed
inside the window thread with nux::WindowThread::WatchFdForEvents. Only
read-data is supported right now.

When there is new data available to be read, the provided FdWatchCallback will
be called by the window thread, and executed inside the window thread. From
there, clients can safely modify data contained by the WindowThread object.

Remove a watched file descriptor with UnwatchFd.

It is safe to add file descriptors for watching during the call to the provided
ThreadUserInitFunc in CreateGUIThread. This is because the main loop
implementation must be created before those file descriptors are added for
watching (particularly in the case of GLib, because nux::WindowThread::InitGlibLoop
will use the default GMainContext the first time it is called, and then create
a new one for each window thereafter.

Added three new tests to demonstrate and cover this functionality.

[==========] Running 3 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 3 tests from TestWindowThread
[ RUN ] TestWindowThread.WatchFd
[ OK ] TestWindowThread.WatchFd (3213 ms)
[ RUN ] TestWindowThread.MultiWatchFd
[ OK ] TestWindowThread.MultiWatchFd (3163 ms)
[ RUN ] TestWindowThread.OneFdEvent
[ OK ] TestWindowThread.OneFdEvent (3155 ms)
[----------] 3 tests from TestWindowThread (9532 ms total)

[----------] Global test environment tear-down
[==========] 3 tests from 1 test case ran. (9532 ms total)
[ PASSED ] 3 tests.

Fixed segfaulting test xtest-text-input. That test unsafely mutated object
state contained by the WindowThread running in another thread inside of the
test thread. This resulted in a null pointer dereference because it had a
side effect of calling nux::GetWindowThread () which relies on up-to-date
thread-local-storage. That test now defines a protocol to communicate to
the running program to mutate the relevant state, and then communicate back
when it has completed execution.

(LP: #1097281)

Implement a message-passing system in nux::WindowThread and nux::ProgramFramework.

This makes it possible to define and implement protocols for WindowThreads to
communicate with each other in a thread-safe way without having to resort to
using window system events or the like.

New file descriptors can be added for watching with a watch-callback executed
inside the window thread with nux::WindowThread::WatchFdForEvents. Only
read-data is supported right now.

When there is new data available to be read, the provided FdWatchCallback will
be called by the window thread, and executed inside the window thread. From
there, clients can safely modify data contained by the WindowThread object.

Remove a watched file descriptor with UnwatchFd.

It is safe to add file descriptors for watching during the call to the provided
ThreadUserInitFunc in CreateGUIThread. This is because the main loop
implementation must be created before those file descriptors are added for
watching (particularly in the case of GLib, because nux::WindowThread::InitGlibLoop
will use the default GMainContext the first time it is called, and then create
a new one for each window thereafter.

Added three new tests to demonstrate and cover this functionality.

[==========] Running 3 tests from 1 test case.
[----------] Global test environment set-up.
[----------] 3 tests from TestWindowThread
[ RUN ] TestWindowThread.WatchFd
[ OK ] TestWindowThread.WatchFd (3213 ms)
[ RUN ] TestWindowThread.MultiWatchFd
[ OK ] TestWindowThread.MultiWatchFd (3163 ms)
[ RUN ] TestWindowThread.OneFdEvent
[ OK ] TestWindowThread.OneFdEvent (3155 ms)
[----------] 3 tests from TestWindowThread (9532 ms total)

[----------] Global test environment tear-down
[==========] 3 tests from 1 test case ran. (9532 ms total)
[ PASSED ] 3 tests.

Fixed segfaulting test xtest-text-input. That test unsafely mutated object
state contained by the WindowThread running in another thread inside of the
test thread. This resulted in a null pointer dereference because it had a
side effect of calling nux::GetWindowThread () which relies on up-to-date
thread-local-storage. That test now defines a protocol to communicate to
the running program to mutate the relevant state, and then communicate back
when it has completed execution.

(LP: #1097281)

I understand that introducing such a large amount of framework is a hefty change for what appears to be fixing a small bug. However, I feel that it is justified. I'm surprised that as a toolkit Nux didn't even have support for this in the first place, since it is definitely a valid usecase where the underlying framework is controlling program flow. There also didn't appear to be any other (sane and safe) way of implementing the calls to SetKeyFocusArea properly, and I don't know if the test works without them.

Note: xtest-text-entry-deadkeys still fails with this:
Nux: TextEntry created: Ok
X Error of failed request: BadValue (integer parameter out of range for operation)
  Major opcode of failed request: 132 (XTEST)
  Minor opcode of failed request: 2 (X_XTestFakeInput)
  Value in failed request: 0x0
  Serial number of failed request: 276
  Current serial number in output stream: 278

That's a separate issue, which I'll deal with in another review (bug 1112321)

[Brandon Schaefer (brandontschaefer)]

Looks good to me, the idea of it sounds excellent as well. Im just a bit rusty on fd/thread and pipes and would love a seconds reviewer.

review: Approve

[Marco Trevisan (Treviño) (3v1n0)]

It seems mostly ok to me...

[PS Jenkins bot (ps-jenkins)]

FAILED: Autolanding.
More details in the following jenkins job:
http://jenkins.qa.ubuntu.com/job/nux-mbs-autolanding/112/
Executed test runs:
    FAILURE: http://jenkins.qa.ubuntu.com/job/nux-mbs-autolanding/./build=panda-pbuilder,distribution=raring,flavor=armhf/112/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/nux-mbs-autolanding/./build=pbuilder,distribution=raring,flavor=amd64/112/console
    FAILURE: http://jenkins.qa.ubuntu.com/job/nux-mbs-autolanding/./build=pbuilder,distribution=raring,flavor=i386/112/console

review: Needs Fixing (continuous-integration)

[Francis Ginther (fginther)]

Jenkins job failed due to a configuration issue. Re-approving.

=== modified file 'Nux/MainLoopGLib.cpp'
--- Nux/MainLoopGLib.cpp	2012-12-08 01:03:38 +0000
+++ Nux/MainLoopGLib.cpp	2013-02-01 13:11:21 +0000
@@ -9,6 +9,7 @@
 #include "FloatingWindow.h"
 
 #include "WindowThread.h"
+#include "MainLoopGLib.h"
 
 namespace nux
 {
@@ -296,9 +297,104 @@
     }
   }
 
+  namespace
+  {
+    typedef struct _ExternalNuxSource
+    {
+      GSource source;
+      GPollFD pfd;
+    } ExternalNuxSource;
+
+    gboolean ExternalSourcePrepareFunc(GSource *source, gint *timeout)
+    {
+      /* Always block for new events */
+      *timeout = -1;
+      return FALSE;
+    }
+
+    gboolean ExternalSourceCheckFunc(GSource *source)
+    {
+      /* Only return true if there are events waiting for us */
+      ExternalNuxSource *extSource =
+          reinterpret_cast <ExternalNuxSource *>(source);
+      return extSource->pfd.revents == G_IO_IN;
+    }
+
+    gboolean ExternalSourceDispatchFunc(GSource     *source,
+                                        GSourceFunc callback,
+                                        gpointer    user_data)
+    {
+      return (*callback) (user_data);
+    }
+
+    static GSourceFuncs externalGLibFuncs =
+    {
+      &ExternalSourcePrepareFunc,
+      &ExternalSourceCheckFunc,
+      &ExternalSourceDispatchFunc, 
+      NULL,
+      /* Technically we shouldn't be touching these, but the compiler
+       * will complain if we don't */
+      0,
+      0
+    };
+  }
+
+  gboolean WindowThread::ExternalSourceCallback(gpointer user_data)
+  {
+    WindowThread::ExternalFdData *data =
+      reinterpret_cast <WindowThread::ExternalFdData *> (user_data);
+
+    data->cb ();
+    return TRUE;
+  }
+
+  void ExternalGLibSources::AddFdToGLibLoop(gint fd,
+                                            gpointer data,
+                                            GSourceFunc callback,
+                                            GMainContext *context)
+  {
+    GSource *source = g_source_new(&externalGLibFuncs, sizeof (ExternalNuxSource));
+    ExternalNuxSource *extSource =
+      reinterpret_cast <ExternalNuxSource *> (source);
+    extSource->pfd.fd = fd;
+    extSource->pfd.events = G_IO_IN;
+    extSource->pfd.revents = 0;
+
+    g_source_add_poll (source, &extSource->pfd);
+    g_source_set_callback (source, callback, data, NULL);
+
+    g_source_attach (source, context);
+  }
+
+  void ExternalGLibSources::RemoveFdFromGLibLoop(gpointer data)
+  {
+    g_source_remove_by_user_data(data);
+  }
+
+  void WindowThread::AddFdToGLibLoop(int fd,
+                                     gpointer data,
+                                     GSourceFunc callback)
+  {
+    external_glib_sources_->AddFdToGLibLoop(fd, data, callback, main_loop_glib_context_);
+  }
+
+  void WindowThread::RemoveFdFromGLibLoop(gpointer data)
+  {
+    external_glib_sources_->RemoveFdFromGLibLoop(data);
+  }
+
   void WindowThread::CleanupGlibLoop()
   {
     g_source_remove_by_funcs_user_data(&event_funcs, this);
+
+    for (std::list<ExternalFdData>::iterator it = _external_fds.begin();
+         it != _external_fds.end();
+         ++it)
+    {
+      gpointer data = reinterpret_cast<gpointer>(&(*it));
+      external_glib_sources_->RemoveFdFromGLibLoop(data);
+    }
   }
 
   unsigned int WindowThread::AddGLibTimeout(unsigned int duration)

=== added file 'Nux/MainLoopGLib.h'
--- Nux/MainLoopGLib.h	1970-01-01 00:00:00 +0000
+++ Nux/MainLoopGLib.h	2013-02-01 13:11:21 +0000
@@ -0,0 +1,21 @@
+#ifndef _NUX_MAIN_LOOP_GLIB_H
+#define _NUX_MAIN_LOOP_GLIB_H
+
+#include <glib.h>
+
+#if (defined(NUX_OS_LINUX) || defined(NUX_USE_GLIB_LOOP_ON_WINDOWS)) && (!defined(NUX_DISABLE_GLIB_LOOP))
+
+namespace nux
+{
+  class ExternalGLibSources
+  {
+    public:
+
+      void AddFdToGLibLoop(gint fd, gpointer data, GSourceFunc callback, GMainContext *context);
+      void RemoveFdFromGLibLoop(gpointer data);
+  };
+}
+
+#endif
+
+#endif

=== modified file 'Nux/ProgramFramework/ProgramTemplate.cpp'
--- Nux/ProgramFramework/ProgramTemplate.cpp	2012-09-26 06:44:12 +0000
+++ Nux/ProgramFramework/ProgramTemplate.cpp	2013-02-01 13:11:21 +0000
@@ -17,6 +17,13 @@
  * Authored by: Jay Taoko <[e-mail]>
  *
  */
+#ifndef _GNU_SOURCE
+#define _GNU_SOURCE
+#endif
+
+#include <fcntl.h>
+#include <unistd.h>
+#include <poll.h>
 
 #include "Nux.h"
 #include "VLayout.h"
@@ -25,6 +32,58 @@
 #include "TextEntry.h"
 #include "ProgramTemplate.h"
 
+namespace
+{
+unsigned int MAX_MESSAGE_LEN = 1024;
+
+std::string ReadMessageOnFd(int fd)
+{
+  char buf[MAX_MESSAGE_LEN + 1];
+  ssize_t bytes = read(fd, reinterpret_cast <void *>(buf), MAX_MESSAGE_LEN);
+
+  if (bytes == -1)
+    throw std::runtime_error(std::string("read: ") + strerror(errno));
+
+  buf[bytes] = '\0';
+
+  return std::string(buf);
+}
+
+bool CheckForMessageOnFd(int fd, const std::string &msg)
+{
+  struct pollfd pfd;
+  pfd.events = POLLIN;
+  pfd.revents = 0;
+  pfd.fd = fd;
+
+  if (poll(&pfd, 1, -1) == -1)
+    throw std::runtime_error(std::string("poll: ") + strerror(errno));
+
+  if (ReadMessageOnFd(fd) == msg)
+    return true;
+  else
+    return false;
+}
+
+void WaitForMessageOnFd(int fd, const std::string &msg)
+{
+  while (!CheckForMessageOnFd(fd, msg));
+}
+
+void SendMessageToFd(int fd, const std::string &msg)
+{
+  if (msg.size() > MAX_MESSAGE_LEN)
+  {
+    std::stringstream ss;
+
+    ss << "Only messages up to " << MAX_MESSAGE_LEN << " chars are supported.";
+    throw std::runtime_error(ss.str ());
+  }
+
+  if (write(fd, reinterpret_cast <void *>(const_cast<char *>(msg.c_str())), msg.size()) == -1)
+    throw std::runtime_error(std::string("write: ") + strerror(errno));
+}
+}
 
 ProgramTemplate::ProgramTemplate(const char* program_name,
   int window_width,
@@ -51,6 +110,12 @@
     // Minimum life span is 1 second.
     program_life_span_ = 1000;
   }
+
+  if (pipe2(test_pipe_, O_CLOEXEC) == -1)
+    throw std::runtime_error(strerror(errno));
+
+  if (pipe2(program_pipe_, O_CLOEXEC) == -1)
+    throw std::runtime_error(strerror(errno));
 }
 
 ProgramTemplate::~ProgramTemplate()
@@ -62,7 +127,8 @@
 void ProgramTemplate::Startup()
 {
   nux::NuxInitialize(0);
-  window_thread_ = nux::CreateGUIThread(program_name_.c_str(), window_width_, window_height_, NULL, NULL, NULL);
+  window_thread_ = nux::CreateGUIThread(program_name_.c_str(), window_width_, window_height_, NULL,
+                                        ProgramTemplate::ThreadInitializer, this);
 
   window_thread_->window_configuration.connect(sigc::mem_fun(this, &ProgramTemplate::WaitForConfigureEvent));
 }
@@ -96,6 +162,41 @@
   window_thread_->Run(NULL);
 }
 
+void ProgramTemplate::SendMessageToProgram(const std::string &msg)
+{
+  SendMessageToFd(program_pipe_[1], msg);
+}
+
+void ProgramTemplate::WaitForMessageFromProgram(const std::string &msg)
+{
+  WaitForMessageOnFd(test_pipe_[0], msg);
+}
+
+void ProgramTemplate::SendMessageToTest(const std::string &msg)
+{
+  SendMessageToFd(test_pipe_[1], msg);
+}
+
+void ProgramTemplate::MessageReceivedFromTest()
+{
+  HandleProgramMessage(ReadMessageOnFd(program_pipe_[0]));
+}
+
+void ProgramTemplate::ThreadInitializer(nux::NThread *, void *data)
+{
+  using namespace std::placeholders;
+
+  /* Monitor the test-to-app fd for messages */
+  ProgramTemplate *pt = reinterpret_cast<ProgramTemplate *>(data);
+  pt->window_thread_->WatchFdForEvents(pt->program_pipe_[0],
+                                       std::bind(&ProgramTemplate::MessageReceivedFromTest,
+                                                 pt));
+}
+
+void ProgramTemplate::HandleProgramMessage(const std::string &msg)
+{
+}
+
 bool ProgramTemplate::ReadyToGo()
 {
   return window_thread_;

=== modified file 'Nux/ProgramFramework/ProgramTemplate.h'
--- Nux/ProgramFramework/ProgramTemplate.h	2012-01-27 22:37:13 +0000
+++ Nux/ProgramFramework/ProgramTemplate.h	2013-02-01 13:11:21 +0000
@@ -39,8 +39,22 @@
 
   bool ReadyToGo();
 
+  //!< Use with caution
+  //
+  // This WindowThread here will be valid, but if you call it from
+  // another thread, and the call has a side-effect of calling
+  // nux::GetWindowThread, it will likely result in undefined behaviour
+  // since nux::GetWindowThread depends on the same thread that
+  // GetWindowThread () here provides being the currently running
+  // thread, as nux::GetWindowThread operates in terms of thread
+  // local storage.
   nux::WindowThread* GetWindowThread();
 
+  //!< Tells the program to do something, depending on the defined protocol
+  // implemented in HandleProgramMessage
+  void SendMessageToProgram (const std::string &msg);
+  void WaitForMessageFromProgram (const std::string &msg);
+
 public:
   std::string program_name_;
   int program_life_span_;                  //!< The program will auto-terminate after a delay in milliseconds.
@@ -50,10 +64,22 @@
   int window_width_;
   int window_height_;
 
+protected:
+
+  void SendMessageToTest(const std::string &msg);
+
 private:
+
+  void MessageReceivedFromTest();
+  static void ThreadInitializer(nux::NThread *, void *);
+
+  virtual void HandleProgramMessage(const std::string &);
+
   void ProgramExitCall(void* data);
   void WaitForConfigureEvent(int x, int y, int width, int height);
   bool ready_to_go_;
+  int  program_pipe_[2];
+  int  test_pipe_[2];
 };
 
 #endif // PROGRAMTEMPLATE_H

=== modified file 'Nux/WindowThread.cpp'
--- Nux/WindowThread.cpp	2012-11-20 22:13:38 +0000
+++ Nux/WindowThread.cpp	2013-02-01 13:11:21 +0000
@@ -30,6 +30,7 @@
 #include "FloatingWindow.h"
 
 #include "WindowThread.h"
+#include "MainLoopGLib.h"
 
 namespace nux
 {
@@ -57,6 +58,9 @@
     , embedded_window_(false)
     , window_size_configuration_event_(false)
     , force_rendering_(false)
+#if (defined(NUX_OS_LINUX) || defined(NUX_USE_GLIB_LOOP_ON_WINDOWS)) && (!defined(NUX_DISABLE_GLIB_LOOP))
+    , external_glib_sources_(new ExternalGLibSources)
+#endif
 #ifdef NUX_GESTURES_SUPPORT
     , geis_adapter_(new GeisAdapter)
 #endif
@@ -412,6 +416,10 @@
       return 1;
     }
 
+    // Setup the main loop first, so that user_init_func can add new sources
+    // before running it
+    SetupMainLoop();
+
     if (user_init_func_ && (m_WidgetInitialized == false))
     {
       (*user_init_func_) (this, initialization_data_);
@@ -421,29 +429,42 @@
     return MainLoop();
   }
 
-  int WindowThread::MainLoop()
+  void WindowThread::SetupMainLoop()
   {
     if (IsEmbeddedWindow())
     {
       window_compositor_->FormatRenderTargets(graphics_display_->GetWindowWidth(), graphics_display_->GetWindowHeight());
       InitGlibLoop();
-      RunGlibLoop();
-      return 0;
     }
     else
     {
-      graphics_display_->ShowWindow();
+#if (defined(NUX_OS_LINUX) || defined(NUX_USE_GLIB_LOOP_ON_WINDOWS)) && (!defined(NUX_DISABLE_GLIB_LOOP))
+      InitGlibLoop();
+#endif
     }
+
     // Called the first time so we can initialize the size of the render targets
     // At this stage, the size of the window is known.
     window_compositor_->FormatRenderTargets(graphics_display_->GetWindowWidth(), graphics_display_->GetWindowHeight());
+  }
+
+  int WindowThread::MainLoop()
+  {
+    if (IsEmbeddedWindow())
+    {
+      RunGlibLoop();
+      return 0;
+    }
+    else
+    {
+      graphics_display_->ShowWindow();
+    }
 
     while (GetThreadState() != THREADSTOP)
     {
       if (GetThreadState() == THREADRUNNING)
       {
 #if (defined(NUX_OS_LINUX) || defined(NUX_USE_GLIB_LOOP_ON_WINDOWS)) && (!defined(NUX_DISABLE_GLIB_LOOP))
-        InitGlibLoop();
         RunGlibLoop();
 #else
         ExecutionLoop();
@@ -557,12 +578,58 @@
       graphics_display_->GetSystemEvent(&event);
       int result = DoProcessEvent(event);
 #endif
+
+#if defined(NUX_OS_LINUX)
+      // select on all of our external fds. We can't use poll () here because
+      // ExectionLoop is designed to be a busy-wait system. In reality we should
+      // probably just drop support for this codepath.
+      //
+      // See https://bugs.launchpad.net/nux/+bug/1111216
+
+      fd_set read_fds;
+
+      FD_ZERO(&read_fds);
+
+      for (std::list<ExternalFdData>::iterator it =
+             _external_fds.begin();
+           it != _external_fds.end();
+           ++it)
+      {
+        nuxAssertMsg(it->fd < FD_SETSIZE, "[nux::WindowThread::ExecutionLoop]"\
+                     " file descriptor overflow, aborting");
+        FD_SET(it->fd, &read_fds);
+      }
+
+      // Wait 10 us
+      struct timeval timeout;
+      timeout.tv_sec = 0;
+      timeout.tv_usec = 10;
+
+      int select_result = select(_external_fds.size(), &read_fds, NULL, NULL,
+                                 &timeout);
+
+      nuxAssertMsg(select_result != -1, strerror ("select()");
+
+      // Dispatch any active external fds
+      for (std::list<ExternalFdData>::iterator it =
+             _external_fds.begin();
+           it != _external_fds.end();
+           ++it)
+      {
+        if (FD_ISSET(it->fd, &read_fds))
+          it->cb();
+      }
+
       if (result != 1)
         return result;
     }
 
     return 1;
   }
+#else
+  nuxAssertMsg(_external_fds.empty(), "[nux::WindowThread::ExecutionLoop]"\
+               " external fd support is not implemented on Windows");
+#endif // NUX_OS_LINUX
 #endif // GLIB loop or not
 
   unsigned int WindowThread::ProcessEvent(Event &event)
@@ -1604,6 +1671,42 @@
     return discard_event;
   }
 
+  bool
+  WindowThread::FindDataByFd(const WindowThread::ExternalFdData &data, int fd)
+  {
+    return data.fd == fd;
+  }
+
+  void WindowThread::WatchFdForEvents(int fd, const WindowThread::FdWatchCallback &cb)
+  {
+    ExternalFdData data;
+    data.fd = fd;
+    data.cb = cb;
+
+    _external_fds.push_back(data);
+#if (defined(NUX_OS_LINUX) || defined(NUX_USE_GLIB_LOOP_ON_WINDOWS)) && (!defined(NUX_DISABLE_GLIB_LOOP))
+    AddFdToGLibLoop(fd,
+                    reinterpret_cast <gpointer> (&_external_fds.back()),
+                    WindowThread::ExternalSourceCallback);
+#endif
+  }
+
+  void WindowThread::UnwatchFd(int fd)
+  {
+    using namespace std::placeholders;
+
+    std::list<ExternalFdData>::iterator it =
+      std::find_if (_external_fds.begin(), _external_fds.end(),
+                    std::bind(FindDataByFd, _1, fd));
+
+    if (it != _external_fds.end())
+    {
+#if (defined(NUX_OS_LINUX) || defined(NUX_USE_GLIB_LOOP_ON_WINDOWS)) && (!defined(NUX_DISABLE_GLIB_LOOP))
+      RemoveFdFromGLibLoop(&(*it));
+#endif
+      _external_fds.erase (it);
+    }
+  }
 
   GraphicsDisplay& WindowThread::GetGraphicsDisplay() const
   {

=== modified file 'Nux/WindowThread.h'
--- Nux/WindowThread.h	2012-11-12 20:59:56 +0000
+++ Nux/WindowThread.h	2013-02-01 13:11:21 +0000
@@ -49,7 +49,9 @@
   class Area;
   struct ClientAreaDraw;
 
+
 #if (defined(NUX_OS_LINUX) || defined(NUX_USE_GLIB_LOOP_ON_WINDOWS)) && (!defined(NUX_DISABLE_GLIB_LOOP))
+  class ExternalGLibSources;
   gboolean nux_event_dispatch(GSource *source, GSourceFunc callback, gpointer user_data);
   gboolean nux_timeout_dispatch(gpointer user_data);
 #endif
@@ -332,6 +334,10 @@
     GeisAdapter *GetGeisAdapter() const {return geis_adapter_.get();}
 #endif
 
+    typedef std::function<void()> FdWatchCallback;
+    void WatchFdForEvents(int fd, const FdWatchCallback &);
+    void UnwatchFd(int fd);
+
   protected:
 
     /*!
@@ -474,6 +480,8 @@
      */
     unsigned int DoProcessEvent(Event &event);
 
+    void SetupMainLoop();
+
     //! Execute the main loop of this thread.
     /*!
         Execute the main loop of this thread.
@@ -609,6 +617,18 @@
     */
     std::map<int, EventInspectorStorage> _event_inspectors_map; //!< map of events inspectors
 
+    typedef struct _ExternalFdData
+    {
+      int             fd;
+      FdWatchCallback cb;
+    } ExternalFdData;
+
+    //! List of external sources. This might make more sense as a map,
+    //  but providing a struct provides us much nicer GLib integration
+    std::list<ExternalFdData> _external_fds;
+
+    static bool FindDataByFd(const WindowThread::ExternalFdData &data, int fd);
+
 #if (defined(NUX_OS_LINUX) || defined(NUX_USE_GLIB_LOOP_ON_WINDOWS)) && (!defined(NUX_DISABLE_GLIB_LOOP))
     GMainLoop *main_loop_glib_;
     GMainContext *main_loop_glib_context_;
@@ -616,12 +636,18 @@
     friend gboolean nux_timeout_dispatch(gpointer user_data);
     std::list<GSource*> child_window_list_;
 
+    std::unique_ptr<ExternalGLibSources> external_glib_sources_;
+
     void InitGlibLoop();
     void RunGlibLoop();
     void StopGLibLoop();
     void CleanupGlibLoop();
+    void AddFdToGLibLoop(int, gpointer, GSourceFunc);
+    void RemoveFdFromGLibLoop(gpointer);
     bool AddChildWindowGlibLoop(WindowThread* wnd_thread);
 
+    static gboolean ExternalSourceCallback(gpointer user_data);
+
     unsigned int AddGLibTimeout(unsigned int duration);
 #else // no GLIB loop
     Event input_event_;

=== modified file 'tests/gtest-nux-windowthread.cpp'
--- tests/gtest-nux-windowthread.cpp	2012-02-24 05:40:23 +0000
+++ tests/gtest-nux-windowthread.cpp	2013-02-01 13:11:21 +0000
@@ -1,3 +1,10 @@
+#ifndef _GNU_SOURCE
+#define _GNU_SOURCE
+#endif
+#include <fcntl.h>
+#include <unistd.h>
+#include <sys/poll.h>
+
 #include <string>
 #include <fstream>
 
@@ -10,23 +17,265 @@
 
 
 using namespace testing;
+using ::testing::Invoke;
 
 namespace {
 
-TEST(TestWindowThread, TestCreate)
-{
-  nux::NuxInitialize(0);
-  nux::WindowThread *wnd_thread = nux::CreateNuxWindow("Nux Window", 300, 200,
-    nux::WINDOWSTYLE_NORMAL, NULL, false, NULL, NULL);
-
+class MockFdDispatch
+{
+  public:
+    MOCK_METHOD0(dispatch, void());
+
+    MockFdDispatch(int fd) :
+      mFd (fd)
+    {
+      ON_CALL(*this, dispatch())
+        .WillByDefault(Invoke(this, &MockFdDispatch::ClearStream));
+    }
+
+  private:
+
+    void ClearStream()
+    {
+      char buf[2];
+      if (read(mFd, reinterpret_cast <void *>(buf), 2) == -1)
+        throw std::runtime_error(strerror(errno));
+    }
+
+    int mFd;
+};
+
+class TestWindowThread : public ::testing::Test
+{
+  public:
+
+    TestWindowThread()
+      : seconds_to_quit(0)
+      , quit_thread (NULL)
+    {
+      nux::NuxInitialize(0);
+      wnd_thread.reset (nux::CreateNuxWindow("Nux Window", 300, 200,
+                                             nux::WINDOWSTYLE_NORMAL, NULL,
+                                             false, TestWindowThread::WindowThreadInit,
+                                             this));
+
+      if (pipe2(quit_pipe_, O_CLOEXEC) == -1)
+        throw std::runtime_error(strerror(errno));
+
+      if (pipe2(test_quit_pipe_, O_CLOEXEC) == -1)
+        throw std::runtime_error(strerror(errno));
+
+      WatchFdForEventsOnRun(quit_pipe_[0],
+                            std::bind(&TestWindowThread::QuitMessage,
+                                      this));
+
+
+    }
+
+    ~TestWindowThread()
+    {
+      if (quit_thread)
+        delete quit_thread;
+
+      close(quit_pipe_[0]);
+      close(quit_pipe_[1]);
+      close(test_quit_pipe_[0]);
+      close(test_quit_pipe_[1]);
+    }
+
+  protected:
+
+    std::unique_ptr <nux::WindowThread> wnd_thread;
+    unsigned int                        seconds_to_quit;
+    nux::SystemThread                   *quit_thread;
+
+    void WatchFdForEventsOnRun(int fd,
+                               const nux::WindowThread::FdWatchCallback &cb)
+    {
+      WatchedFd wfd;
+      wfd.fd = fd;
+      wfd.callback = cb;
+
+      watched_fds.push_back(wfd);
+    }
+
+    /* Watchdog threads are really not that great, but ping-ponging between
+     * the window and application threads would over-complicate this test */
+    void QuitAfter(unsigned int seconds)
+    {
+      seconds_to_quit = seconds;
+      quit_thread = nux::CreateSystemThread(NULL, TestWindowThread::QuitTask, this);
+      quit_thread->Start(NULL);
+    }
+
+    void WaitForQuit()
+    {
+      struct pollfd pfd;
+
+      pfd.events = POLLIN;
+      pfd.revents = 0;
+      pfd.fd = test_quit_pipe_[0];
+
+      if (poll(&pfd, 1, -1) == -1)
+        throw std::runtime_error(strerror(errno));
+
+      /* Something was written to the read end of our pipe,
+       * we can continue now */
+    }
+
+  private:
+
+    typedef struct _WatchedFd
+    {
+      int                                fd;
+      nux::WindowThread::FdWatchCallback callback;
+    } WatchedFd;
+
+    static void WindowThreadInit(nux::NThread *, void *data)
+    {
+      TestWindowThread *tw = reinterpret_cast <TestWindowThread *>(data);
+
+      /* Monitor all nominated fd's */
+      for (std::vector<WatchedFd>::iterator it = tw->watched_fds.begin();
+           it != tw->watched_fds.end();
+           ++it)
+      {
+        tw->wnd_thread->WatchFdForEvents(it->fd, it->callback);
+      }
+    }
+
+    static void QuitTask(nux::NThread *, void *data)
+    {
+      TestWindowThread *tw = reinterpret_cast <TestWindowThread *> (data);
+
+      nux::SleepForMilliseconds(tw->seconds_to_quit * 1000);
+
+      const char *buf = "w\0";
+      if (write(tw->quit_pipe_[1],
+                reinterpret_cast<void *>(const_cast<char *>(buf)),
+                2) == -1)
+        throw std::runtime_error(strerror(errno));
+    }
+
+    void QuitMessage()
+    {
+      wnd_thread->ExitMainLoop();
+
+      /* Send a message indicating that the main loop
+       * exited and we can safely continue */
+      const char *buf = "w\0";
+      if (write(test_quit_pipe_[1],
+                reinterpret_cast<void *>(const_cast<char *>(buf)),
+                2) == -1)
+        throw std::runtime_error(strerror(errno));
+    }
+
+    std::vector<WatchedFd> watched_fds;
+
+    int quit_pipe_[2];
+    int test_quit_pipe_[2];
+};
+
+TEST_F(TestWindowThread, Create)
+{
   ASSERT_TRUE(wnd_thread != NULL);
   EXPECT_EQ(wnd_thread->GetWindowTitle(), std::string("Nux Window"));
   EXPECT_EQ(wnd_thread->IsModalWindow(), false);
 
   EXPECT_EQ(wnd_thread->IsComputingLayout(), false);
   EXPECT_EQ(wnd_thread->IsInsideLayoutCycle(), false);
-
-  delete wnd_thread;
+}
+
+TEST_F(TestWindowThread, WatchFd)
+{
+  int pipefd[2];
+
+  if (pipe2 (pipefd, O_CLOEXEC) == -1)
+    throw std::runtime_error(strerror(errno));
+
+  MockFdDispatch dispatch(pipefd[0]);
+
+  EXPECT_CALL(dispatch, dispatch());
+
+  WatchFdForEventsOnRun(pipefd[0], std::bind(&MockFdDispatch::dispatch, &dispatch));
+  const char *buf = "w\0";
+  if (write(pipefd[1], reinterpret_cast <void *> (const_cast <char *> (buf)), 2) == -1)
+    throw std::runtime_error (strerror(errno));
+
+  QuitAfter(3);
+  wnd_thread->Run(NULL);
+
+  /* We must wait for quit before closing the pipes */
+  WaitForQuit();
+
+  close (pipefd[0]);
+  close (pipefd[1]);
+}
+
+TEST_F(TestWindowThread, MultiWatchFd)
+{
+  int pipefd[2], pipefd2[2];
+
+  if (pipe2 (pipefd, O_CLOEXEC) == -1)
+    throw std::runtime_error(strerror(errno));
+  if (pipe2 (pipefd2, O_CLOEXEC) == -1)
+    throw std::runtime_error(strerror(errno));
+
+  MockFdDispatch dispatch(pipefd[0]), dispatch2(pipefd2[0]);
+
+  EXPECT_CALL(dispatch, dispatch());
+  EXPECT_CALL(dispatch2, dispatch());
+
+  WatchFdForEventsOnRun(pipefd[0], std::bind(&MockFdDispatch::dispatch, &dispatch));
+  WatchFdForEventsOnRun(pipefd2[0], std::bind(&MockFdDispatch::dispatch, &dispatch2));
+  const char *buf = "w\0";
+  if (write(pipefd[1], reinterpret_cast <void *> (const_cast <char *> (buf)), 2) == -1)
+    throw std::runtime_error (strerror(errno));
+  if (write(pipefd2[1], reinterpret_cast <void *> (const_cast <char *> (buf)), 2) == -1)
+    throw std::runtime_error (strerror(errno));
+
+  QuitAfter(3);
+  wnd_thread->Run(NULL);
+
+  /* We must wait for quit before closing the pipes */
+  WaitForQuit();
+
+  close (pipefd[0]);
+  close (pipefd[1]);
+  close (pipefd2[0]);
+  close (pipefd2[1]);
+}
+
+TEST_F(TestWindowThread, OneFdEvent)
+{
+  int pipefd[2], pipefd2[2];
+
+  if (pipe2 (pipefd, O_CLOEXEC) == -1)
+    throw std::runtime_error(strerror(errno));
+  if (pipe2 (pipefd2, O_CLOEXEC) == -1)
+    throw std::runtime_error(strerror(errno));
+
+  MockFdDispatch dispatch(pipefd[0]), dispatch2(pipefd2[0]);
+
+  EXPECT_CALL(dispatch, dispatch());
+  EXPECT_CALL(dispatch2, dispatch()).Times(0);
+
+  WatchFdForEventsOnRun(pipefd[0], std::bind(&MockFdDispatch::dispatch, &dispatch));
+  WatchFdForEventsOnRun(pipefd2[0], std::bind(&MockFdDispatch::dispatch, &dispatch2));
+  const char *buf = "w\0";
+  if (write(pipefd[1], reinterpret_cast <void *> (const_cast <char *> (buf)), 2) == -1)
+    throw std::runtime_error (strerror(errno));
+
+  QuitAfter(3);
+  wnd_thread->Run(NULL);
+
+  /* We must wait for quit before closing the pipes */
+  WaitForQuit();
+
+  close (pipefd[0]);
+  close (pipefd[1]);
+  close (pipefd2[0]);
+  close (pipefd2[1]);
 }
 
 

=== modified file 'tests/xtest-text-entry.cpp'
--- tests/xtest-text-entry.cpp	2012-10-01 23:37:25 +0000
+++ tests/xtest-text-entry.cpp	2013-02-01 13:11:21 +0000
@@ -29,6 +29,12 @@
 #include <X11/keysym.h> 
 #include "nux_automated_test_framework.h"
 
+namespace
+{
+std::string RESET_KEY_FOCUS_AREA_MSG = "r";
+std::string KEY_FOCUS_AREA_RESET_MSG = "d";
+}
+
 class TextTextEntry: public ProgramTemplate
 {
 public:
@@ -41,12 +47,19 @@
   void OnActivated();
   void OnCursorMoved(int);
   void ResetEvents();
+  void SendResetKeyFocusAreaMessageToProgram();
+
 
   nux::TextEntry* text_entry_;
 
   bool clicked_;
   bool activated_;
   bool cursor_moved_;
+
+private:
+
+  /* Handled inside the TextEntry WindowThread */
+  void HandleProgramMessage(const std::string &);
 };
 
 TextTextEntry::TextTextEntry(const char* program_name,
@@ -74,6 +87,22 @@
   cursor_moved_ = false;
 }
 
+void TextTextEntry::SendResetKeyFocusAreaMessageToProgram()
+{
+  SendMessageToProgram(RESET_KEY_FOCUS_AREA_MSG);
+  WaitForMessageFromProgram(KEY_FOCUS_AREA_RESET_MSG);
+}
+
+void TextTextEntry::HandleProgramMessage(const std::string &m)
+{
+  if (m == RESET_KEY_FOCUS_AREA_MSG)
+  {
+    GetWindowThread()->GetWindowCompositor().SetKeyFocusArea(NULL);
+    GetWindowThread()->GetWindowCompositor().SetKeyFocusArea(text_entry_);
+    SendMessageToTest(KEY_FOCUS_AREA_RESET_MSG);
+  }
+}
+
 void TextTextEntry::TextEntryClick(nux::TextEntry* text_entry)
 {
   if (text_entry_ == text_entry)
@@ -185,8 +214,7 @@
     test.ViewSendString("Nux");
     test.TestReportMsg(test_textentry->text_entry_->GetText() == "Nux", "Typed \"Nux\"");
 
-    test_textentry->GetWindowThread()->GetWindowCompositor().SetKeyFocusArea(NULL); 
-    test_textentry->GetWindowThread()->GetWindowCompositor().SetKeyFocusArea(test_textentry->text_entry_);
+    test_textentry->SendResetKeyFocusAreaMessageToProgram();
 
     test_textentry->ResetEvents();
     test.ViewSendLeft();
@@ -205,8 +233,7 @@
     test.ViewSendString("Nux");
     test.TestReportMsg(test_textentry->text_entry_->GetText() == "Nux", "Typed \"Nux\"");
 
-    test_textentry->GetWindowThread()->GetWindowCompositor().SetKeyFocusArea(NULL); 
-    test_textentry->GetWindowThread()->GetWindowCompositor().SetKeyFocusArea(test_textentry->text_entry_);
+    test_textentry->SendResetKeyFocusAreaMessageToProgram();
 
     test_textentry->ResetEvents();
     test.ViewSendReturn();