Merge lp:~drizzle-developers/drizzle/replication-docs-7.1 into lp:~drizzle-trunk/drizzle/development

[Brian Aker]

[Sekhar (msekharmca)]

Please elaborate the content..

Thanks,
Sekhar

=== modified file 'docs/administration/authentication.rst'
--- docs/administration/authentication.rst	2011-11-14 22:29:19 +0000
+++ docs/administration/authentication.rst	2012-03-18 00:36:19 +0000
@@ -1,3 +1,5 @@
+.. _authentication:
+
 Authentication
 ==============
 

=== modified file 'docs/administration/authorization.rst'
--- docs/administration/authorization.rst	2011-11-14 22:29:19 +0000
+++ docs/administration/authorization.rst	2012-03-18 00:36:19 +0000
@@ -1,3 +1,5 @@
+.. _authorization:
+
 Authorization
 =============
 

=== renamed file 'docs/administration/getting_started.rst' => 'docs/administration/index.rst'
--- docs/administration/getting_started.rst	2011-10-23 16:01:37 +0000
+++ docs/administration/index.rst	2012-03-18 00:36:19 +0000
@@ -1,5 +1,7 @@
-Getting Started
-===============
+.. _administering_drizzle:
+
+Administering Drizzle
+=====================
 
 Drizzle is a unique database server because most of its functionality is
 provided by plugins.  In fact, the primary job of :program:`drizzled`,

=== modified file 'docs/administration/plugins.rst'
--- docs/administration/plugins.rst	2012-03-03 02:42:42 +0000
+++ docs/administration/plugins.rst	2012-03-18 00:36:19 +0000
@@ -1,5 +1,7 @@
 .. program:: drizzled
 
+.. _plugins:
+
 Plugins
 =======
 
@@ -24,7 +26,7 @@
  * :ref:`connection_id_plugin` - Return the current connection_id (connection_id)
  * console_plugin - Console Client (console) (TODO: documentation missing)
  * :ref:`crc32_plugin` - CRC32 Function (crc32)
- * :ref:`default_replicator_plugin` - Default Replicator (default_replicator)
+ * :ref:`default_replicator` - Default Replicator (default_replicator)
  * :ref:`drizzle_protocol_plugin` - Drizzle Protocol (drizzle_protocol)
  * :ref:`errmsg_stderr_plugin` - Error Messages to stderr (errmsg_stderr)
  * :ref:`error_dictionary_plugin` - Data Dictionary for Errors. (error_dictionary)

=== modified file 'docs/clients/drizzledump.rst'
--- docs/clients/drizzledump.rst	2012-03-08 00:54:48 +0000
+++ docs/clients/drizzledump.rst	2012-03-18 00:36:19 +0000
@@ -1,3 +1,5 @@
+.. _drizzledump:
+
 drizzledump Backup Tool
 =======================
 

=== modified file 'docs/configuration/drizzled.rst'
--- docs/configuration/drizzled.rst	2012-03-08 00:54:48 +0000
+++ docs/configuration/drizzled.rst	2012-03-18 00:36:19 +0000
@@ -15,6 +15,10 @@
 General Options
 ^^^^^^^^^^^^^^^
 
+.. option:: --daemon, -d
+
+   Run :program:`drizzled` as a daemon.
+
 .. option:: --help, -?
 
    Display this help and exit.
@@ -23,6 +27,20 @@
 
    Display this help and exit after initializing plugins.
 
+.. option:: --user, -u ARG
+
+   :Default:
+   :Variable: 
+
+   Run drizzled daemon as user.
+
+.. option:: --version, -V
+
+   :Default:
+   :Variable: ``version``
+
+   Print the version of Drizzle and exit.
+
 .. _drizzled_config_file_options:
 
 Config File Options
@@ -93,6 +111,25 @@
 
      --plugin_remove=syslog,md5
 
+.. _drizzled_replication_options:
+
+Replication Options
+^^^^^^^^^^^^^^^^^^^
+
+.. option:: --replicate-query
+
+   :Default:
+   :Variable: ``replicate_query``
+   
+   Include the SQL query in replicated protobuf messages.
+
+.. option:: --transaction-message-threshold
+
+   :Default: 1048576
+   :Variable: ``transaction_message_threshold``
+
+   Max message size written to transaction log, valid values 131072 - 1048576 bytes.
+
 .. _drizzled_kernel_options:
 
 Kernel Options
@@ -421,13 +458,6 @@
    A global constraint for read-rnd-buffer-size for all clients, cannot be set
    lower than --read-rnd-buffer-size.  Setting to 0 means unlimited.
 
-.. option:: --replicate-query
-
-   :Default:
-   :Variable: ``replicate_query``
-   
-   Include the SQL query in replicated protobuf messages.
-
 .. option:: --scheduler ARG
 
    :Default: multi-thread
@@ -539,27 +569,6 @@
 
    Default transaction isolation level.
 
-.. option:: --transaction-message-threshold
-
-   :Default: 1048576
-   :Variable: ``transaction_message_threshold``
-
-   Max message size written to transaction log, valid values 131072 - 1048576 bytes.
-
-.. option:: --user, -u ARG
-
-   :Default:
-   :Variable: 
-
-   Run drizzled daemon as user.
-  
-.. option:: --version, -V
-
-   :Default:
-   :Variable: ``version``
-
-   Output version information and exit.
-
 .. _drizzled_variables:
 
 Variables

=== modified file 'docs/configuration/options.rst'
--- docs/configuration/options.rst	2011-11-14 22:29:19 +0000
+++ docs/configuration/options.rst	2012-03-18 00:36:19 +0000
@@ -129,8 +129,8 @@
 ^^^^^^^^^^^^^^^^^^^^
 
 Command line options have the form ``--option-name=value`` (the ``=`` is
-optional).  This form works for both :ref:`drizzled_options` and all
-plugin options.  For example::
+optional).  This form works for :ref:`drizzled options <drizzled_options>`
+and all plugin options.  For example::
 
    drizzled --basedir=/opt/drizzle --innodb.buffer-pool-size=500M
 

=== modified file 'docs/functions/datetime.rst'
--- docs/functions/datetime.rst	2012-03-14 19:16:59 +0000
+++ docs/functions/datetime.rst	2012-03-18 00:36:19 +0000
@@ -29,7 +29,11 @@
 |statement_timestamp()	            |timestamp with time zone*	          |Current date and time (start of current statement)               |
 +-----------------------------------+-------------------------------------+-----------------------------------------------------------------+
 
+<<<<<<< TREE
 Drizzle timezone is always UTC.
+=======
+*Drizzle timezone is always UTC*.
+>>>>>>> MERGE-SOURCE
 
 Extract
 -------

=== modified file 'docs/index.rst'
--- docs/index.rst	2012-01-30 21:09:19 +0000
+++ docs/index.rst	2012-03-18 00:36:19 +0000
@@ -73,7 +73,7 @@
 .. toctree::
    :maxdepth: 2
    
-   administration/getting_started
+   administration/index
    administration/drizzled
    administration/authentication
    administration/authorization
@@ -85,8 +85,13 @@
 -----------
 .. toctree::
    :maxdepth: 2
+   :glob:
 
-   replication/drizzle
+   replication/index
+   replication/replicators/index
+   replication/appliers/index
+   replication/messages/index
+   replication/examples/index
 
 SQL Language
 ------------

=== added directory 'docs/replication/appliers'
=== added file 'docs/replication/appliers/index.rst'
--- docs/replication/appliers/index.rst	1970-01-01 00:00:00 +0000
+++ docs/replication/appliers/index.rst	2012-03-18 00:36:19 +0000
@@ -0,0 +1,33 @@
+.. _appliers:
+
+.. _replication_appliers:
+
+Appliers
+========
+
+Appliers are the other end point of a
+:ref:`replication stream <replication_streams>`, the first being
+:ref:`replicators`.  Appliers provide an
+interface between replicators and a service.  The service can be anything:
+another Drizzle server, a different database server, a message queue, etc.
+Appliers receive replication events from replicators and apply them to the
+service, although the term "apply" is used loosely.  Appliers can do anything
+with replication events that provides useful behavior for the service.
+For example, an applier may write the replicaiton event to a file on disk,
+or it may send it over the network to some other service to be processed.
+
+Appliers are implemented by plugins and specify the unique name of a replicator with which Drizzle should pair it to create a replicaiton stream.  Applier plugins default to using the :ref:`default_replicator`, but they can be configured to use another replicator.
+
+Most applier plugins are loaded and ran on the :ref:`originating_server`,
+but this is not a requirement.  For example, the :ref:`slave_applier` is
+loaded on one server (the slave) and connects to and pairs with the
+:ref:`default_replicator` on another server (the master).
+
+Drizzle includes the following applier plugins:
+
+.. toctree::
+   :maxdepth: 1
+
+   slave
+   rabbitmq
+   zeromq

=== added directory 'docs/replication/appliers/rabbitmq'
=== added file 'docs/replication/appliers/rabbitmq.rst'
--- docs/replication/appliers/rabbitmq.rst	1970-01-01 00:00:00 +0000
+++ docs/replication/appliers/rabbitmq.rst	2012-03-18 00:36:19 +0000
@@ -0,0 +1,199 @@
+.. _rabbitmq_applier:
+
+RabbitMQ Applier
+================
+
+The RabbitMQ applier plugin, named ``rabbitmq``, applies replication events to a `RabbitMQ <http://www.rabbitmq.com>`_ server.  This can be used to create advanced replication solutions, to visualize data, or to build triggers.
+
+Loading
+-------
+
+To load this plugin, start :program:`drizzled` with::
+
+   --plugin-add=rabbitmq
+
+Loading the plugin may not enable or configure it.  See the plugin's
+:ref:`rabbitmq_configuration` and :ref:`rabbitmq_variables`.
+
+.. seealso:: :ref:`drizzled_plugin_options` for more information about adding and removing plugins.
+
+.. _rabbitmq_configuration:
+
+Configuration
+-------------
+
+These command line options configure the plugin when :program:`drizzled`
+is started.  See :ref:`command_line_options` for more information about
+specifying command line options.
+
+.. program:: drizzled
+
+.. option:: --rabbitmq.exchange ARG
+
+   :Default: ReplicationExchange
+   :Variable: :ref:`rabbitmq_exchange <rabbitmq_exchange>`
+
+   Name of RabbitMQ exchange to publish to
+
+.. option:: --rabbitmq.host ARG
+
+   :Default: localhost
+   :Variable: :ref:`rabbitmq_host <rabbitmq_host>`
+
+   Host name to connect to
+
+.. option:: --rabbitmq.password ARG
+
+   :Default: guest
+   :Variable: :ref:`rabbitmq_password <rabbitmq_password>`
+
+   RabbitMQ password
+
+.. option:: --rabbitmq.port ARG
+
+   :Default: 5672
+   :Variable: :ref:`rabbitmq_port <rabbitmq_port>`
+
+   Port to connect to
+
+.. option:: --rabbitmq.routingkey ARG
+
+   :Default: ReplicationRoutingKey
+   :Variable: :ref:`rabbitmq_routingkey <rabbitmq_routingkey>`
+
+   Name of RabbitMQ routing key to use
+
+.. option:: --rabbitmq.use-replicator ARG
+
+   :Default: default_replicator
+   :Variable:
+
+   Name of the replicator plugin to use (default='default_replicator')
+
+.. option:: --rabbitmq.username ARG
+
+   :Default: guest
+   :Variable: :ref:`rabbitmq_username <rabbitmq_username>`
+
+   RabbitMQ username
+
+.. option:: --rabbitmq.virtualhost ARG
+
+   :Default: /
+   :Variable: :ref:`rabbitmq_virtualhost <rabbitmq_virtualhost>`
+
+   RabbitMQ virtualhost
+
+.. _rabbitmq_variables:
+
+Variables
+---------
+
+These variables show the running configuration of the plugin.
+See `variables` for more information about querying and setting variables.
+
+.. _rabbitmq_exchange:
+
+* ``rabbitmq_exchange``
+
+   :Scope: Global
+   :Dynamic: No
+   :Option: :option:`--rabbitmq.exchange`
+
+   Name of RabbitMQ exchange to publish to
+
+.. _rabbitmq_host:
+
+* ``rabbitmq_host``
+
+   :Scope: Global
+   :Dynamic: No
+   :Option: :option:`--rabbitmq.host`
+
+   Host name to connect to
+
+.. _rabbitmq_password:
+
+* ``rabbitmq_password``
+
+   :Scope: Global
+   :Dynamic: No
+   :Option: :option:`--rabbitmq.password`
+
+   RabbitMQ password
+
+.. _rabbitmq_port:
+
+* ``rabbitmq_port``
+
+   :Scope: Global
+   :Dynamic: No
+   :Option: :option:`--rabbitmq.port`
+
+   Port to connect to
+
+.. _rabbitmq_routingkey:
+
+* ``rabbitmq_routingkey``
+
+   :Scope: Global
+   :Dynamic: No
+   :Option: :option:`--rabbitmq.routingkey`
+
+   Name of RabbitMQ routing key to use
+
+.. _rabbitmq_username:
+
+* ``rabbitmq_username``
+
+   :Scope: Global
+   :Dynamic: No
+   :Option: :option:`--rabbitmq.username`
+
+   RabbitMQ username
+
+.. _rabbitmq_virtualhost:
+
+* ``rabbitmq_virtualhost``
+
+   :Scope: Global
+   :Dynamic: No
+   :Option: :option:`--rabbitmq.virtualhost`
+
+   RabbitMQ virtualhost
+
+
+Implementation Details
+----------------------
+
+* :program:`drizzled` will not sart if the rabbitmq server is not available.
+* If the rabbitmq server goes away, the plugin will try to reconnect and resend the message 3 times, after that, the transaction is rolled back.
+
+.. _rabbitmq_version:
+
+Version
+-------
+
+This documentation applies to :ref:`rabbitmq_0.1_drizzle_7.0`.
+
+To see which version of the ``rabbitmq`` plugin a Drizzle server is running,
+execute:
+
+.. code-block:: mysql
+
+   SELECT MODULE_VERSION FROM DATA_DICTIONARY.MODULES WHERE MODULE_NAME='rabbitmq'
+
+Changelog
+---------
+
+.. toctree::
+   :maxdepth: 2
+
+   rabbitmq/changelog
+
+.. _rabbitmq_authors:
+
+Authors
+-------
+
+Marcus Eriksson

=== added file 'docs/replication/appliers/rabbitmq/changelog.rst'
--- docs/replication/appliers/rabbitmq/changelog.rst	1970-01-01 00:00:00 +0000
+++ docs/replication/appliers/rabbitmq/changelog.rst	2012-03-18 00:36:19 +0000
@@ -0,0 +1,13 @@
+.. _rabbitmq_changelog:
+
+rabbitmq Changelog
+==================
+
+.. _rabbitmq_0.1_drizzle_7.0:
+
+rabbitmq 0.1 (Drizzle 7.0)
+--------------------------
+
+:Released: March 15, 2011
+
+* First release.

=== added directory 'docs/replication/appliers/slave'
=== added file 'docs/replication/appliers/slave.rst'
--- docs/replication/appliers/slave.rst	1970-01-01 00:00:00 +0000
+++ docs/replication/appliers/slave.rst	2012-03-18 00:36:19 +0000
@@ -0,0 +1,64 @@
+.. _slave:
+
+.. _slave_applier:
+
+Slave Applier
+=============
+
+The slave applier plugin, named ``slave``, provides a native implementation
+of replication between Drizzle servers by applying replication events from
+multiple "master" servers to a "slave" server.  A slave is a server on which
+the ``slave`` plugin is loaded, configured, and ran.  Slaves can connect to up
+to ten masters at once over the network and pair with the
+:ref:`default_replicator` on each them.  Slaves apply every replication
+event from every master; conequently, they contain the same schemas and
+data as their masters.
+
+Replication chains are supported: slaves can also be masters and replicate
+events to other slaves.  There is no limit to the number of levels in which
+masters and slaves can be enchained.
+
+Replication circles, or master-master replication, are not currently supported.
+Replication will break on one of the masters if this is attempted.
+
+The next two sections, :ref:`slave_configuration` and :ref:`slave_administration`,  contain necessary information for configuring and administering slave-based replication.  The third section, :ref:`slave_details`, covers technical details about how the ``slave`` plugin and slave-based replication work.
+
+.. toctree::
+   :maxdepth: 2
+
+   slave/configuration
+   slave/administration
+   slave/details
+
+.. _slave_version:
+
+Version
+-------
+
+This documentation applies to :ref:`slave_1.1_drizzle_7.1`.
+
+To see which version of the ``slave`` plugin a Drizzle server is running,
+execute:
+
+.. code-block:: mysql
+
+   SELECT MODULE_VERSION FROM DATA_DICTIONARY.MODULES WHERE MODULE_NAME='slave'
+
+Changelog
+---------
+
+.. toctree::
+   :maxdepth: 2
+
+   slave/changelog
+
+.. _slave_authors:
+
+Authors
+-------
+
+David Shrewsbury
+   Original ``slave`` plugin code.  Multi-master code.
+
+Daniel Nichter
+   Documentation for :ref:`slave_1.1_drizzle_7.1`.

=== added file 'docs/replication/appliers/slave/administration.rst'
--- docs/replication/appliers/slave/administration.rst	1970-01-01 00:00:00 +0000
+++ docs/replication/appliers/slave/administration.rst	2012-03-18 00:36:19 +0000
@@ -0,0 +1,332 @@
+.. _slave_administration:
+
+.. _slave_admin:
+
+Slave Administration
+********************
+
+.. _slave_threads:
+
+Slave Threads
+=============
+
+The :ref:`slave` plugin uses two threads per master to read and apply replication events:
+
+* An IO (or producer) thread
+* An applier (or consumer) thread
+
+The IO thread connects to a master and reads replication events from the :ref:`slave_innodb_transaction_log` on the master.  The applier thread applies the replication events from the IO thread to the slave.  In other words, the IO thread produces replication events from the master, and the applier thread consumes and applies them to the slave.
+
+There is currently no way to stop, start, or restart the threads.  Unless a replication error occurs, the threads should function continuously and remain in :ref:`RUNNING <slave_thread_running_status>` status.  If a replication error does occur, the Drizzle server must be stopped, the error fixed, and the server restarted.
+
+.. _slave_thread_statuses:
+
+Statuses
+--------
+
+Slave thread statuses are indicated by the ``status`` columns of the :ref:`sys_replication_tables`.  A slave thread status is always one of these values:
+
+.. _slave_thread_running_status:
+
+RUNNING
+   The thread is working properly, applying replicaiton events.
+
+.. _slave_thread_stopped_status:
+
+STOPPED
+   The thread has stopped due to an error; replication events are not being applied.
+
+.. _sys_replication_tables:
+
+sys_replication Tables
+======================
+
+The ``sys_replication`` schema on the slave has tables with information about the state of :ref:`slave_threads` and other replication-related information:
+
+.. code-block:: mysql
+
+   drizzle> SHOW TABLES FROM sys_replication;
+   +---------------------------+
+   | Tables_in_sys_replication |
+   +---------------------------+
+   | applier_state             | 
+   | io_state                  | 
+   | queue                     | 
+   +---------------------------+
+
+If the ``sys_replication`` table is not available, then the :ref:`slave` plugin failed to load.
+
+.. _sys_replication_io_state:
+
+io_state
+--------
+
+``sys_replication.io_state`` contains the state of IO threads, one per row:
+
+.. code-block:: mysql
+
+   drizzle> SELECT * FROM sys_replication.io_state;
+   *************************** 1. row ***************************
+   master_id: 1
+      status: RUNNING
+   error_msg: 
+
+master_id
+   ID (number) of the master to which the thread is connected.  The number corresponds to the the master number in the :ref:`slave_config_file`.
+
+status
+   :ref:`Status <slave_thread_statuses>` of the IO thread.
+
+error_msg
+   Error message explaining why the thread has :ref:`STOPPED <slave_thread_statuses>`.
+
+.. _sys_replication_applier_state:
+
+applier_state
+-------------
+
+``sys_replication.applier_state`` contains the state of applier threads, one per row:
+
+.. code-block:: mysql
+
+   drizzle> SELECT * FROM sys_replication.applier_state\G
+   *************************** 1. row ***************************
+                 master_id: 1
+    last_applied_commit_id: 18
+   originating_server_uuid: 9908C6AA-A982-4763-B9BA-4EF5F933D219
+     originating_commit_id: 18
+                    status: RUNNING
+                 error_msg: 
+
+master_id
+   ID (number) of the master from which the thread is applying replication events.  The number corresponds to the the master number in the :ref:`slave_config_file`.
+
+last_applied_commit_id
+   Value of the ``COMMIT_ID`` from the master's replication log of the most recently executed transaction.  See definition of the data_dictionary.sys_replication_log table.
+
+originating_server_uuid
+   UUID of the :ref:`originating_server`.
+
+originating_commit_id
+   ``COMMIT_ID`` from the :ref:`originating_server`.
+
+status
+   :ref:`Status <slave_thread_statuses>` of the applier thread.
+
+error_msg
+   Error message explaining why the thread has :ref:`STOPPED <slave_thread_stopped_status>`.
+
+.. _sys_replication_queue:
+
+queue
+-----
+
+``sys_replication.io_state`` contains replication events that have not yet been applied by the applier thread, one per row:
+
+.. code-block:: mysql
+
+   drizzle> SELECT * FROM sys_replication.queue\G
+   *************************** 1. row ***************************
+                    trx_id: 925
+                    seg_id: 1
+              commit_order: 12
+   originating_server_uuid: 9908C6AA-A982-4763-B9BA-4EF5F933D219
+     originating_commit_id: 12
+                       msg: transaction_context {
+      server_id: 1
+      transaction_id: 925
+      start_timestamp: 1330211976689868
+      end_timestamp: 1330211976689874
+   }
+   statement {
+      type: DROP_SCHEMA
+      start_timestamp: 1330211976689872
+      end_timestamp: 1330211976689873
+      drop_schema_statement {
+         schema_name: "foo"
+      }
+   }
+   segment_id: 1
+   end_segment: true
+                 master_id: 1
+
+.. _slave_lag:
+
+Slave Lag
+=========
+
+Slaves intentionally lag masters by at least :ref:`applier-thread-sleep <slave_cfg_common_options>` or :ref:`io-thread-sleep <slave_cfg_common_options>` seconds.  Since :ref:`replication_events` are transactions, *slave lag is the number of committed transactions behind a master*.  To determine a slave's lag, first query the last committed transaction ID on the master:
+
+.. code-block:: mysql
+
+   drizzle> SELECT MAX(COMMIT_ID) FROM DATA_DICTIONARY.SYS_REPLICATION_LOG;
+   +----------------+
+   | MAX(COMMIT_ID) |
+   +----------------+
+   |              6 | 
+   +----------------+
+
+Then query the last applied transaction ID on the slave:
+
+.. code-block:: mysql
+
+   drizzle> SELECT MAX(last_applied_commit_id) FROM sys_replication.applier_state;
+   +-----------------------------+
+   | MAX(last_applied_commit_id) |
+   +-----------------------------+
+   |                           6 | 
+   +-----------------------------+
+
+In this example, the slave is caught up to the master because both the last commited transaction ID on the master and the last applied transaction ID on the slave are 6.
+
+If the last applied transaction ID on a slave is less than the last committed transaciton ID on a master, then the slave lags the master by *N transactions* (not seconds) where N is the difference of the two transaction ID values.
+
+Master Connections
+==================
+
+Slaves connect to masters like normal users by specifying a username and password (see the :ref:`slave_cfg_master_options`).  Therefore, slave connections on a master are visible in the master's processlist and sessions, but they are not specially indicated.  If the :ref:`slave_user_account` uses slave-specific usernames like "slave1", then the slave connections can be viewed like:
+
+.. code-block:: mysql
+
+   drizzle> SELECT * FROM DATA_DICTIONARY.PROCESSLIST  WHERE USERNAME LIKE 'slave%'\G
+   *************************** 1. row ***************************
+                ID: 2
+          USERNAME: slave
+              HOST: 127.0.0.1
+                DB: NULL
+           COMMAND: Sleep
+              TIME: 0
+             STATE: NULL
+              INFO: NULL
+   HAS_GLOBAL_LOCK: 0
+
+The ``DATA_DICTIONARY.SESSIONS`` table can be queried similarly:
+
+.. code-block:: mysql
+
+   drizzle> SELECT * FROM DATA_DICTIONARY.SESSIONS WHERE SESSION_USERNAME LIKE 'slave%'\G
+   *************************** 1. row ***************************
+         SESSION_ID: 2
+   SESSION_USERNAME: slave1
+       SESSION_HOST: 127.0.0.1
+    SESSION_CATALOG: LOCAL
+     SESSION_SCHEMA: NULL
+            COMMAND: Sleep
+              STATE: NULL
+              QUERY: NULL
+    HAS_GLOBAL_LOCK: 0
+     IS_INTERACTIVE: 0
+         IS_CONSOLE: 0
+
+Or, slave connections can be viewed by specifying the slave server's hostname, like:
+
+.. code-block:: mysql
+
+   drizzle> SELECT * FROM DATA_DICTIONARY.PROCESSLIST  WHERE HOSTNAME = '192.168.1.5'\G
+   *************************** 1. row ***************************
+                ID: 3
+          USERNAME: slave
+              HOST: 192.168.1.5
+                DB: NULL
+           COMMAND: Sleep
+              TIME: 0
+             STATE: NULL
+              INFO: NULL
+   HAS_GLOBAL_LOCK: 0
+
+.. _slave_innodb_transaction_log:
+
+InnoDB Transaction Log
+======================
+
+The :ref:`slave` requires the :ref:`innodb_transaction_log` on the master to retrieve replication events.  This transaction log is stored as an internal table within InnoDB, but there are two tables which provide access to its data:
+
+* ``DATA_DICTIONARY.SYS_REPLICATION_LOG``
+* ``DATA_DICTIONARY.INNODB_REPLICATION_LOG``
+
+The :ref:`IO thread <slave_threads>` from a slave (which connects to a master) reads transactions (replicaiton events) directly from ``DATA_DICTIONARY.SYS_REPLICATION_LOG``.  The transaction messages are binary which makes the table data unreadable by most humans:
+
+.. code-block:: mysql
+
+   drizzle> SELECT * from DATA_DICTIONARY.SYS_REPLICATION_LOG\G
+   *************************** 1. row ***************************
+                        ID: 772
+                     SEGID: 1
+                 COMMIT_ID: 1
+             END_TIMESTAMP: 1331841800496546
+   ORIGINATING_SERVER_UUID: 98ECEA09-BA65-489D-9382-F8D15098B1AE
+     ORIGINATING_COMMIT_ID: 1
+               MESSAGE_LEN: 33
+                   MESSAGE: ?????????
+
+The last column, ``MESSAGE``, contains the actual transaction data that the client renders as question marks because the data is binary, not text.
+
+The ``DATA_DICTIONARY.INNODB_REPLICATION_LOG`` table contains the same data as the ``DATA_DICTIONARY.SYS_REPLICATION_LOG`` table, but it converts the transaction data to text:
+
+.. code-block:: mysql
+
+   drizzle> SELECT * from DATA_DICTIONARY.INNODB_REPLICATION_LOG\G
+   *************************** 1. row ***************************
+               TRANSACTION_ID: 772
+       TRANSACTION_SEGMENT_ID: 1
+                    COMMIT_ID: 1
+                END_TIMESTAMP: 1331841800496546
+      ORIGINATING_SERVER_UUID: 98ECEA09-BA65-489D-9382-F8D15098B1AE
+        ORIGINATING_COMMIT_ID: 1
+   TRANSACTION_MESSAGE_STRING: transaction_context {
+      server_id: 1
+      transaction_id: 772
+      start_timestamp: 1331841800496542
+      end_timestamp: 1331841800496546
+   }
+   event {
+      type: STARTUP
+   }
+   segment_id: 1
+   end_segment: true
+           TRANSACTION_LENGTH: 33
+
+The ``TRANSACTION_MESSAGE_STRING`` column contains the text representation of the ``MESSAGE`` column from the ``DATA_DICTIONARY.SYS_REPLICATION_LOG`` table.
+
+
+``DATA_DICTIONARY.INNODB_REPLICATION_LOG`` is read-only, but ``DATA_DICTIONARY.SYS_REPLICATION_LOG`` can be modified which allows the transaction log to be maintained, as described in the next section.
+
+Transaction Log Maintenance
+---------------------------
+
+Currently, the InnoDB transaction log grows without bounds and old transactions are never deleted.  The InnoDB transaction log must be maintained manually by carefully deleting old transactions that are no longer needed from the ``DATA_DICTIONARY.SYS_REPLICATION_LOG`` table.
+
+.. warning:: Care must be taken to avoid deleting transactions that slaves have not yet applied else data will be lost and replication will break.
+
+Follow these steps to trim the InnoDB transaction log without affecting slave function:
+
+#. Query each slave for the ``last_applied_commit_id`` value from the :ref:`sys_replication.applier_state <sys_replication_applier_state>` table.
+#. Choose the **minimum** value obtained from step one. This is the marker value for the slave that is the furthest behind the master.
+#. Using the marker value from the previous step, delete rows from ``DATA_DICTIONARY.SYS_REPLICATION_LOG`` that have a ``COMMIT_ID`` less than the marker value.
+
+For example, if there are two slaves, query each one for the minimum ``last_applied_commit_id``:
+
+.. code-block:: mysql
+
+   slave1> SELECT last_applied_commit_id FROM sys_replicaiton.applier_state;
+   +------------------------+
+   | last_applied_commit_id |
+   +------------------------+
+   |                   3000 | 
+   +------------------------+
+
+   slave2> SELECT last_applied_commit_id FROM sys_replicaiton.applier_state;
+   +------------------------+
+   | last_applied_commit_id |
+   +------------------------+
+   |                   2877 | 
+   +------------------------+
+
+slave2 has the smallest value for ``last_applied_commit_id``, 2877, so this value is the marker for deleting records from the master's transaction log:
+
+.. code-block:: mysql
+
+  master> DELETE FROM DATA_DICTIONARY.SYS_REPLICATION_LOG
+       -> WHERE COMMIT_ID < 2877;
+
+This permanently deletes all old, unneeded records from the InnoDB transaction log.

=== added file 'docs/replication/appliers/slave/changelog.rst'
--- docs/replication/appliers/slave/changelog.rst	1970-01-01 00:00:00 +0000
+++ docs/replication/appliers/slave/changelog.rst	2012-03-18 00:36:19 +0000
@@ -0,0 +1,27 @@
+.. _slave_changelog:
+
+slave Changelog
+===============
+
+.. _slave_1.1_drizzle_7.1:
+
+slave 1.1 (Drizzle 7.1)
+-----------------------
+
+:Released:
+
+* Added multi-master-support (https://blueprints.launchpad.net/drizzle/+spec/slave-multi-master).
+
+  * Changed all options, except ``--slave.config-file``, from command line options to slave config file options.
+  * Added ``ignore-errors`` option for slave config file. 
+
+* Rewrote :ref:`slave` docs (https://blueprints.launchpad.net/drizzle/+spec/docs71-focus-areas).
+
+.. _slave_1.0_drizzle_7.0:
+
+slave 1.0 (Drizzle 7.0)
+-----------------------
+
+:Released: March 15, 2011
+
+* First release.

=== added file 'docs/replication/appliers/slave/configuration.rst'
--- docs/replication/appliers/slave/configuration.rst	1970-01-01 00:00:00 +0000
+++ docs/replication/appliers/slave/configuration.rst	2012-03-18 00:36:19 +0000
@@ -0,0 +1,229 @@
+.. program:: drizzled
+
+.. _slave_configuration:
+
+.. _slave_config:
+
+Slave Configuration
+*******************
+
+Configuring a Drizzle :ref:`replication stream <replication_streams>` using the :ref:`slave` requires:
+
+#. Load a :ref:`replicator plugin <replicators>` on the masters
+#. Enabling :option:`--innodb.replication-log` on the masters
+#. Creating user accounts for the slaves on the masters
+#. Writing a :ref:`slave_config_file` for :option:`--slave.config-file` on the slave
+#. Loading and configure the ``slave`` plugin on the slave
+
+The first three steps are performed on a master, and the last two steps are performed on a slave.
+
+.. _configuring_a_master:
+
+Configuring a Master
+====================
+
+A single slave can apply replication events from up to ten masters at once.
+There are three requirements for each master:
+
+#. :ref:`replicators`
+#. :option:`--innodb.replication-log`
+#. :ref:`slave_user_account`
+
+A :ref:`replicator plugin <replicators>` must be loaded on each master.  The :ref:`default_replicator` is loaded by default.  To verify that it is loaded on a master, execute:
+
+.. code-block:: sql
+
+ drizzle> SELECT * FROM DATA_DICTIONARY.PLUGINS WHERE PLUGIN_NAME = 'default_replicator';
+ +--------------------+-----------------------+-----------+--------------------+
+ | PLUGIN_NAME        | PLUGIN_TYPE           | IS_ACTIVE | MODULE_NAME        |
+ +--------------------+-----------------------+-----------+--------------------+
+ | default_replicator | TransactionReplicator |         1 | default_replicator |
+ +--------------------+-----------------------+-----------+--------------------+
+
+If the plugin is not loaded, verify that the server was *not* started with
+``--plugin-remove default_replicator``.  If it was, remove that option and
+restart the server.
+
+To use a different replicator, :ref:`configure Drizzle <configuring_drizzle>` to load the replicator plugin on startup, and specify the replicator name with :option:`--innodb.use-replicator`.
+
+Each master must also be started with :option:`--innodb.replication-log`
+to enable the :ref:`innodb_transaction_log` which is not enabled by default.
+Therefore, Drizzle must be configured with this option at startup.
+See :ref:`configuring_drizzle` for more information.  To verify that the
+InnoDB replication log is active, execute:
+
+.. code-block:: mysql
+
+   drizzle> SELECT * FROM DATA_DICTIONARY.GLOBAL_VARIABLES WHERE VARIABLE_NAME = 'innodb_replication_log';
+   +------------------------+----------------+
+   | VARIABLE_NAME          | VARIABLE_VALUE |
+   +------------------------+----------------+
+   | innodb_replication_log | ON             | 
+   +------------------------+----------------+
+
+   drizzle> SELECT * FROM DATA_DICTIONARY.INNODB_REPLICATION_LOG LIMIT 1;
+   -- The query should return one row showing a replication event.
+
+.. _slave_user_account:
+
+Slave User Account
+------------------
+
+A user account is required on the master for slave connections, unless no :ref:`authentication` is used (which is highly inadvisable).  One user account can be used for all slaves, or individual user accounts can be used for each slave.  In either case, the user account credentials (username and password) for a master are specified in the :ref:`slave_config_file`.
+
+:ref:`authorization` must be configured on the master to allow slave user accounts to access the ``DATA_DICTIONARY`` schema, else the :ref:`slave IO thread <slave_threads>` will fail to start on the slave:
+
+.. code-block:: mysql
+
+   drizzle> SELECT * FROM sys_replication.io_state\G
+   *************************** 1. row ***************************
+   master_id: 1
+      status: STOPPED
+   error_msg: Replication slave: Access denied for user 'slave' to schema 'data_dictionary'
+
+The :ref:`sys_replication_tables` are discussed in :ref:`slave_admin`.
+
+:ref:`authorization` cannot be used to filter which schemas or tables are replicated because slave connections do no access individual schemas and tables, they only access the :ref:`innodb_transaction_log`.  To filter which schemas or tables are replicated, configure the master to use a filtering replicator like :ref:`filtered_replicator`, as described above. 
+
+.. _configuring_a_slave:
+
+Configuring a Slave
+===================
+
+After :ref:`configuring_a_master`, configuring a slave requires only:
+
+#. :ref:`slave_config_file`
+#. :ref:`slave_plugin`
+
+.. _slave_config_file:
+
+Slave Config File
+-----------------
+
+A slave config file is a plain text file that contains connection and configuration options for each master.  At least one master must be specifed, and masters must be numbered sequentially from 1 to 10. The general syntax of a slave config file is:
+
+.. code-block:: ini
+
+ # comment
+ common-option=value
+ [masterN]
+ master-specific-option=value
+
+There are two types of options: common options which apply to all masters, and master-specific options which only apply to the preceding ``[masterN]`` header where ``N`` is the sequentially numbered master, starting with 1.  Whitespace
+before and after lines and around ``=`` (equal signs) is ignored.
+
+The simplest possible slave config file is:
+
+.. code-block:: ini
+
+   [master1]
+   master-host=<master hostname>
+   master-user=slave1
+
+See :ref:`slave_examples` for complete, working examples.
+
+.. _slave_cfg_common_options:
+
+Common Options
+--------------
+
+These options must be specified first, before any ``[masterN]`` headers.
+
+.. confval:: applier-thread-sleep
+
+   :Default: 5
+
+   The number of seconds the applier (consumer) thread sleeps between applying
+   replication events from the local queue.
+
+.. confval:: ignore-errors
+
+   Ignore errors and continue applying replication events.  It is generally
+   a bad idea to use this option!
+
+.. confval:: io-thread-sleep
+
+   :Default: 5
+
+   The number of seconds the IO (producer) thread sleeps between queries to the
+   master for more replication events.
+
+.. confval:: seconds-between-reconnects
+
+   :Default: 30
+
+   The number of seconds to wait between reconnect attempts when the master
+   server becomes unreachable.
+
+.. _slave_cfg_master_options:
+
+Master-specific Options
+-----------------------
+
+These options must be specified after a ``[masterN]`` header.
+
+.. confval:: master-host
+
+   Hostname/IP address of the master server.
+
+.. confval:: master-port
+
+   :Default: 3306
+
+   Drizzle port used by the master server.
+
+.. confval:: master-user
+
+   Username to use for connecting to the master server.
+   See :ref:`slave_user_account`.
+
+.. confval:: master-pass
+
+   Password associated with the username given by :confval:`master-user`.
+   See :ref:`slave_user_account`.
+
+.. program:: drizzledump
+
+.. confval:: max-commit-id
+
+   Maximum commit ID the slave is assumed to have applied from the master.
+   This value will be used by the slave to determine where to begin retrieving
+   replication events from the master transaction log. This option can be used
+   to provision a new slave by setting it to the value output from the
+   :ref:`drizzledump` when used with the :option:`--single-transaction` option.
+
+.. confval:: max-reconnects
+
+   :Default: 10
+
+   The number of reconnection attempts the slave plugin will try if the
+   master server becomes unreachable.
+
+.. _slave_plugin:
+
+slave Plugin
+------------
+
+A slave must load the ``slave`` plugin which is not loaded by default.
+This plugin has only one option:
+
+.. program:: drizzled
+
+.. option:: --slave.config-file FILE
+
+   :Default: :file:`BASEDIR/etc/slave.cfg`
+   :Variable:
+
+   Full path to a :ref:`slave_config_file`.
+   By default, the plugin looks for a file named :file:`slave.cfg`
+   in :file:`BASEDIR/etc/` where :file:`BASEDIR` is determined by
+   :option:`--basedir`.
+
+Since a slave can connect to multiple masters, all other options are set
+in the :ref:`slave_config_file`.  Once a slave config file has been written, start Drizzle with the ``slave`` plugin like:
+
+.. code-block:: bash
+
+  $ drizzled --plugin-add slave --slave.config-file /etc/drizzled/slave.conf
+
+If the masters are configured properly, and the slave config file is correct, and the slave plugin loads successfully, the :ref:`sys_replication_tables` will be accessible on the slave as described in the next topic, :ref:`slave_admin`.

=== added file 'docs/replication/appliers/slave/details.rst'
--- docs/replication/appliers/slave/details.rst	1970-01-01 00:00:00 +0000
+++ docs/replication/appliers/slave/details.rst	2012-03-18 00:36:19 +0000
@@ -0,0 +1,11 @@
+.. _slave_details:
+
+Slave Details
+=============
+
+The following sections dive into the technical aspects of the :ref:`slave`.  In general, it should not be necessary to know this information to :ref:`configure <slave_configuration>` or :ref:`adminster <slave_admin>` a slave.  This information is useful for troubleshooting, developers, hackers, and those who wish to "look under the hood."
+
+Slave Plugin Class
+------------------
+
+Although the documentation for the :ref:`slave` calls the plugin an applier, which implies that the plugin subclasses the TransactionApplier class, the :ref:`slave_plugin` is not in fact a TransactionApplier, it is a Daemon.  The :ref:`innodb_transaction_log` is a TransactionApplier which defaults to using with the :ref:`default_replicator` (see :file:`plugin/innobase/handler/replication_log.h`).

=== added file 'docs/replication/appliers/zeromq.rst'
--- docs/replication/appliers/zeromq.rst	1970-01-01 00:00:00 +0000
+++ docs/replication/appliers/zeromq.rst	2012-03-18 00:36:19 +0000
@@ -0,0 +1,55 @@
+.. _zeromq:
+
+.. _zeromq_applier:
+
+ZeroMQ Applier
+==============
+
+ZeroMQ is a messaging library that allows you to easily build complex
+communication systems. The ZeroMQ plugin allows drizzle to publish
+transactions to a local PUB socket. Many clients can subscribe to
+these transactions. The first frame of the message sent out is the
+schema name the transaction touched - this enables clients to only
+subscribe to the interesting schemas (note that certain "transactions"
+are without a schema, like SET xyz for example, for these, the first
+frame is empty).
+
+Getting started
+---------------
+
+First, install zeromq, get the code from `zeromq.org
+<http://zeromq.org>`_, then you can build drizzle, watch the
+./configure output to verify that drizzle finds the libraries needed.
+
+Now you are good to go, simply start drizzle with --plugin-add=zeromq
+and drizzle will start publishing transactions. The only configuration
+parameter available is:
+
+.. code-block:: bash
+
+  --zeromq.endpoint arg (=tcp://*:9999) - the endpoint to expose.
+
+Now you can write a simple python script to verify that it works,
+something like this will work:
+
+.. code-block:: python
+
+  import zmq
+
+  ctx = zmq.Context()
+  s = ctx.socket(zmq.SUB)
+  s.setsockopt(zmq.SUBSCRIBE, '')
+  s.connect('tcp://localhost:9999')
+  i = 0
+  while True:
+      i = i+1
+      s.recv_multipart()
+      print i
+
+and then you can generate some load:
+
+.. code-block:: bash
+
+  bin/drizzleslap -c 10 -a --auto-generate-sql-add-autoincrement --burnin
+
+which creates 10 threads and generates random queries.

=== removed file 'docs/replication/drizzle.rst'
--- docs/replication/drizzle.rst	2011-10-23 05:45:09 +0000
+++ docs/replication/drizzle.rst	1970-01-01 00:00:00 +0000
@@ -1,324 +0,0 @@
-*******************
-Drizzle Replication
-*******************
-
-Replication events are recorded using messages in the `Google Protocol Buffer
-<http://code.google.com/p/protobuf/>`_ (GPB) format. GPB messages can contain
-sub-messages. There is a single main "envelope" message, Transaction, that
-is passed to plugins that subscribe to the replication stream.
-
-Configuration Options
-#####################
-
-**transaction_message_threshold**
-
-    Controls the size, in bytes, of the Transaction messages. When a Transaction
-    message exceeds this size, a new Transaction message with the same
-    transaction ID will be created to continue the replication events.
-    See :ref:`bulk-operations` below.
-
-
-**replicate_query**
-
-    Controls whether the originating SQL query will be included within each
-    Statement message contained in the enclosing Transaction message. The
-    default global value is FALSE which will not include the query in the
-    messages. It can be controlled per session, as well. For example:
-
-    .. code-block:: mysql
-
-       drizzle> set @@replicate_query = 1;
-
-    The stored query should be used as a guide only, and never executed
-    on a slave to perform replication as this will lead to incorrect results.
-
-Message Definitions
-###################
-
-The GPB messages are defined in .proto files in the drizzled/message
-directory of the Drizzle source code. The primary definition file is
-transaction.proto. Messages defined in this file are related in the
-following ways::
-
-
-  ------------------------------------------------------------------
-  |                                                                |
-  | Transaction message                                            |
-  |                                                                |
-  |   -----------------------------------------------------------  |
-  |   |                                                         |  |
-  |   | TransactionContext message                              |  |
-  |   |                                                         |  |
-  |   -----------------------------------------------------------  |
-  |   -----------------------------------------------------------  |
-  |   |                                                         |  |
-  |   | Statement message 1                                     |  |
-  |   |                                                         |  |
-  |   -----------------------------------------------------------  |
-  |   -----------------------------------------------------------  |
-  |   |                                                         |  |
-  |   | Statement message 2                                     |  |
-  |   |                                                         |  |
-  |   -----------------------------------------------------------  |
-  |                             ...                                |
-  |   -----------------------------------------------------------  |
-  |   |                                                         |  |
-  |   | Statement message N                                     |  |
-  |   |                                                         |  |
-  |   -----------------------------------------------------------  |
-  ------------------------------------------------------------------
-
-with each Statement message looking like so::
-
-  ------------------------------------------------------------------
-  |                                                                |
-  | Statement message                                              |
-  |                                                                |
-  |   -----------------------------------------------------------  |
-  |   |                                                         |  |
-  |   | Common information                                      |  |
-  |   |                                                         |  |
-  |   |  - Type of Statement (INSERT, DELETE, etc)              |  |
-  |   |  - Start Timestamp                                      |  |
-  |   |  - End Timestamp                                        |  |
-  |   |  - (OPTIONAL) Actual SQL query string                   |  |
-  |   |                                                         |  |
-  |   -----------------------------------------------------------  |
-  |   -----------------------------------------------------------  |
-  |   |                                                         |  |
-  |   | Statement subclass message 1 (see below)                |  |
-  |   |                                                         |  |
-  |   -----------------------------------------------------------  |
-  |                             ...                                |
-  |   -----------------------------------------------------------  |
-  |   |                                                         |  |
-  |   | Statement subclass message N (see below)                |  |
-  |   |                                                         |  |
-  |   -----------------------------------------------------------  |
-  ------------------------------------------------------------------
-
-The Transaction Message
-#######################
-
-The main "envelope" message which represents an atomic transaction
-which changed the state of a server is the Transaction message class.
-
-The Transaction message contains two pieces:
-
-#. A TransactionContext message containing information about the
-   transaction as a whole, such as the ID of the executing server,
-   the start and end timestamp of the transaction, segmenting
-   metadata and a unique identifier for the transaction.
-#. A vector of Statement messages representing the distinct SQL
-   statements which modified the state of the server.  The Statement
-   message is, itself, a generic envelope message containing a
-   sub-message which describes the specific data modification which
-   occurred on the server (such as, for instance, an INSERT statement).
-
-The Statement Message
-#####################
-
-The generic "envelope" message containing information common to each
-SQL statement executed against a server (such as a start and end timestamp
-and the type of the SQL statement) as well as a Statement subclass message
-describing the specific data modification event on the server.
-
-Each Statement message contains a type member which indicates how readers
-of the Statement should construct the inner Statement subclass representing
-a data change.
-
-Statements are recorded separately as sometimes individual statements
-have to be rolled back.
-
-
-.. _bulk-operations:
-
-How Bulk Operations Work
-########################
-
-Certain operations which change large volumes of data on a server
-present a specific set of problems for a transaction coordinator or
-replication service. If all operations must complete atomically on a
-publishing server before replicas are delivered the complete
-transactional unit:
-
-#. The publishing server could consume a large amount of memory
-   building an in-memory Transaction message containing all the
-   operations contained  in the entire transaction.
-#. A replica, or subscribing server, is wasting time waiting on the
-   eventual completion (commit) of the large transaction on the
-   publishing server. It could be applying pieces of the large
-   transaction in the meantime...
-
-In order to prevent the problems inherent in (1) and (2) above, Drizzle's
-replication system uses a mechanism which provides bulk change
-operations.
-
-A single transaction in the database can possibly be represented with
-multiple protobuf Transaction messages if the message grows too large.
-This can happen if you have a bulk transaction, or a single statement
-affecting a very large number of rows, or just a large transaction with
-many statements/changes.
-
-For the first two examples, it is likely that the Statement sub-message
-itself will get segmented, causing another Transaction message to be
-created to hold the rest of the Statement's row changes. In these cases,
-it is enough to look at the segment information stored in the Statement
-message (see example below).
-
-For the last example, the Statement sub-messages may or may not be
-segmented, but we could still need to split the individual Statements up into
-multiple Transaction messages to keep the Transaction message size from
-growing too large. In this case, the segment information in the Statement
-submessages is not helpful if the Statement isn't segmented. We need this
-information in the Transaction message itself.
-
-Segmenting a Single SQL Statement
-*********************************
-
-When a regular SQL statement modifies or inserts more rows than a
-certain threshold, Drizzle's replication services component will begin
-sending Transaction messages to replicas which contain a chunk
-(or "segment") of the data which has been changed on the publisher.
-
-When data is inserted, updated, or modified in the database, a
-header containing information about modified tables and fields is
-matched with one or more data segments which contain the actual
-values changed in the statement.
-
-It's easiest to understand this mechanism by following through a real-world
-scenario.
-
-Suppose the following table:
-
-.. code-block:: mysql
-
-  CREATE TABLE test.person
-  (
-    id INT NOT NULL AUTO_INCREMENT PRIMARY KEY
-  , first_name VARCHAR(50)
-  , last_name VARCHAR(50)
-  , is_active CHAR(1) NOT NULL DEFAULT 'Y'
-  );
-
-Also suppose that test.t1 contains 1 million records.
-
-Next, suppose a client issues the SQL statement:
-
-.. code-block:: mysql
-
-  UPDATE test.person SET is_active = 'N';
-
-It is clear that one million records could be updated by this statement
-(we say, "could be" since Drizzle does not actually update a record if
-the UPDATE would not change the existing record...).
-
-In order to prevent the publishing server from having to construct an
-enormous Transaction message, Drizzle's replication services component
-will do the following:
-
-#. Construct a Transaction message with a transaction context containing
-   information about the originating server, the transaction ID, and
-   timestamp information.
-#. Construct an UpdateHeader message with information about the tables
-   and fields involved in the UPDATE statement.  Push this UpdateHeader
-   message onto the Transaction message's statement vector.
-#. Construct an UpdateData message.  Set the *segment_id* member to 1.
-   Set the *end_segment* member to true.
-#. For every record updated in a storage engine, the ReplicationServices
-   component builds a new UpdateRecord message and appends this message
-   to the aforementioned UpdateData message's record vector.
-#. After a certain threshold of records is reached, the
-   ReplicationServices component sets the current UpdateData message's
-   *end_segment* member to false, and proceeds to send the Transaction
-   message to replicators.
-#. The ReplicationServices component then constructs a new Transaction
-   message and constructs a transaction context with the same
-   transaction ID and server information.
-#. A new UpdateData message is created.  The message's *segment_id* is
-   set to N+1 and as new records are updated, new UpdateRecord messages
-   are appended to the UpdateData message's record vector.
-#. While records are being updated, we repeat steps 5 through 7, with
-   only the final UpdateData message having its *end_segment* member set
-   to true.
-
-Segmenting a Transaction
-************************
-
-The Transaction protobuf message also contains *segment_id* member and a
-*end_segment* member. These values are also set appropriately when a
-Statement sub-message is segmented, as described above.
-
-These values are also set when a Transaction must be segmented along
-individual Statement boundaries (i.e., the Statement message itself
-is **not** segmented). In either case, it is enough to check the
-*end_segment* and *segment_id* values of the Transaction message
-to determine if this is a multi-message transaction.
-
-Handling ROLLBACKs
-##################
-
-Both transactions and individual statements may be rolled back.
-
-When a transaction is rolled back, one of two things happen depending
-on whether the transaction is made up of either a single Transaction
-message, or if it is made up of multiple Transaction messages (e.g, bulk
-load).
-
-* For a transaction encapsulated entirely within a single Transaction
-  message, the entire message is simply discarded and not sent through
-  the replication stream.
-* For a transaction which is made up of multiple messages, and at least
-  one message has already been sent through the replication stream, then
-  the Transaction message will contain a Statement message with type =
-  ROLLBACK. This signifies to rollback the entire transaction.
-
-A special Statement message type, ROLLBACK_STATEMENT, is used when
-we have a segmented Statement message (see above) and we need to tell the
-receiver to undo any changes made for this single statement, but not
-for the entire transaction. If the receiver cannot handle rolling back
-a single statement, then a message buffering strategy should be employed 
-to guarantee that a statement was indeed applied successfully before
-executing on the receiver.
-
-.. _replication_streams:
-
-Replication Streams
-###################
-
-The Drizzle kernel handles delivering replication messages to plugins by
-maintaining a list of replication streams. A stream is represented as a
-registered *replicator* and *applier* pair.
-
-When a replication message is generated within the kernel, the replication
-services module of the kernel will send this message to each registered
-*replicator*. The *replicator* will then do something useful with it and
-send it to each *applier* with which it is associated.
-
-Replicators
-***********
-
-A registered *replicator* is a plugin that implements the TransactionReplicator
-API. Each replicator will be plugged into the kernel to receive the Google
-Protobuf messages that are generated as the database is changed. Ideally,
-each registered replicator will transform or modify the messages it receives
-to implement a specific behavior. For example, filtering by schema name.
-
-Each registered replicator should have a unique name. The default replicator,
-cleverly named **default_replicator**, does no transformation at all on the
-replication messages.
-
-Appliers
-********
-
-A registered *applier* is a plugin that implements the TransactionApplier
-API. Appliers are responsible for applying the replication messages that it
-will receive from a registered replicator. The word "apply" is used loosely
-here. An applier may do anything with the replication messages that provides
-useful behavior. For example, an applier may simply write the messages to a
-file on disk, or it may send the messages over the network to some other
-service to be processed.
-
-At the point of registration with the Drizzle kernel, each applier specifies
-the name of a registered replicator that it should be attached to in order to
-make the replication stream pair.

=== added directory 'docs/replication/examples'
=== added file 'docs/replication/examples/index.rst'
--- docs/replication/examples/index.rst	1970-01-01 00:00:00 +0000
+++ docs/replication/examples/index.rst	2012-03-18 00:36:19 +0000
@@ -0,0 +1,32 @@
+.. _replication_examples:
+
+Examples
+========
+
+These replication examples require a working knowledge of:
+
+* :ref:`configuring_drizzle`
+* :ref:`administering_drizzle`
+* :ref:`drizzle_replication`
+
+The examples are complete, but they may not explain every step in detail, and they probably need to be adapted to other environments.  In other words: the examples are guides, not drop-in solutions.  Free and commercial :ref:`help` is available.
+
+.. _slave_examples:
+
+Slave Applier
+-------------
+
+.. toctree::
+   :maxdepth: 1
+   :glob:
+
+   slave/*
+
+RabbitMQ Applier
+----------------
+
+.. toctree::
+   :maxdepth: 1
+   :glob:
+
+   rabbitmq/*

=== added directory 'docs/replication/examples/rabbitmq'
=== added file 'docs/replication/examples/rabbitmq/basic.rst'
--- docs/replication/examples/rabbitmq/basic.rst	1970-01-01 00:00:00 +0000
+++ docs/replication/examples/rabbitmq/basic.rst	2012-03-18 00:36:19 +0000
@@ -0,0 +1,55 @@
+.. _basic_rabbitmq_example:
+
+Basic RabbitMQ
+==============
+
+:Synopsis: Set up replication to a RabbitMQ server
+:Replicator: :ref:`default_replicator`
+:Applier: :ref:`rabbitmq_applier`
+:Version: :ref:`rabbitmq_0.1_drizzle_7.0`
+:Authors: Marcus Eriksson
+
+First install a recent version of RabbitMQ, then install librabbitmq, the C library for talking to the RabbitMQ server:
+
+.. code-block:: bash
+
+   $ hg clone http://hg.rabbitmq.com/rabbitmq-codegen/
+   $ hg clone http://hg.rabbitmq.com/rabbitmq-c/
+   $ cd rabbitmq-c
+   $ autoreconf -f -i
+   $ ./configure
+   $ make
+   $ make install
+
+Now you probably need to rebuild Drizzle since the :program:`rabbitmq` plugin is not built if librabbitmq is not installed.
+
+Finally, start :program:`drizzled` like:
+
+.. code-block:: bash
+
+   sbin/drizzled                       \
+      --daemon                         \
+      --pid-file /var/run/drizzled.pid \
+      --plugin-add rabbitmq            \
+   >> /var/log/drizzled.log 2>&1
+
+To verify that it works, you can start a generic rabbitmq listener from librabbitmq:
+
+.. code-block:: bash
+
+   $ amqp_listen localhost 5672 ReplicationExchange ReplicationRoutingKey
+
+And you should see something like this when you do an INSERT/CREATE/.. (just not a select) in your newly built Drizzle instance::
+
+   Result 0
+   Frame type 1, channel 1
+   Method AMQP_BASIC_DELIVER_METHOD
+   Delivery 1, exchange ReplicationExchange routingkey ReplicationRoutingKey
+
+   00000000: 0A 17 08 01 10 87 36 18 : F0 FA D9 99 FA F1 A7 02  ......6.........
+   00000010: 20 99 81 DA 99 FA F1 A7 : 02 12 40 08 01 10 F2 FA   .........@.....
+   00000020: D9 99 FA F1 A7 02 18 FC : FA D9 99 FA F1 A7 02 2A  ...............*
+   00000030: 17 0A 06 0A 01 62 12 01 : 61 12 06 08 04 12 02 69  .....b..a......i
+   00000040: 64 12 05 08 01 12 01 74 : 32 11 08 01 10 01 1A 0B  d......t2.......
+   00000050: 0A 01 32 0A 02 61 61 10 : 00 10 00 20 01 28 01     ..2..aa.... .(.
+   0000005F:

=== added directory 'docs/replication/examples/slave'
=== added file 'docs/replication/examples/slave/provision_new_slave.rst'
--- docs/replication/examples/slave/provision_new_slave.rst	1970-01-01 00:00:00 +0000
+++ docs/replication/examples/slave/provision_new_slave.rst	2012-03-18 00:36:19 +0000
@@ -0,0 +1,61 @@
+.. _provision_new_slave_example:
+
+Provision New Slave
+===================
+
+:Synopsis: Provision a new slave from a backup of a master
+:Replicator: :ref:`default_replicator`
+:Applier: :ref:`slave`
+:Version: :ref:`slave_1.1_drizzle_7.1`
+:Authors: Marisa Plumb, Kent Bozlinski, Daniel Nichter
+
+The basic formula for creating a new slave for an existing master is:
+
+#. Make a backup of the master databases.
+#. Record the max committed transaction ID of the master at the point the backup was made.
+#. Restore the backup on the new slave.
+#. Start the new slave from the recorded max committed transaction ID of the master.
+
+.. program:: drizzledump
+
+Steps #1 and #2 are performed using the :ref:`drizzledump` and its :option:`--single-transaction` option which prints a comment near the beginning of the dump output with the InnoDB transaction log metadata:
+
+.. code-block:: bash
+
+   master$ drizzledump --all-databases --single-transaction > master-backup.sql
+
+   master$ grep SYS_REPLICATION_LOG master-backup.sql
+   -- SYS_REPLICATION_LOG: COMMIT_ID = 3303, ID = 3500
+
+The ``SYS_REPLICATION_LOG`` comment provides the replication log metadata needed to start a new slave. There are two values:
+
+COMMIT_ID
+   The commit sequence number recorded for the most recently executed transaction stored in the transaction log.  This value is used to determine proper commit order within the log.  ``ID``, the unique transaction identifier, cannot be used because it is assigned when the transaction is started, not when it is committed.
+
+ID
+   The unique transaction identifier associated with the most recently executed transaction stored in the transaction log.
+
+For step #3, start the slave *without* the :ref:`slave_plugin` to prevent it from reading from the master until the backup is imported.  Then import the backup on the slave:
+
+.. code-block:: bash
+
+	slave$ drizzle < master-backup.sql
+
+Stop the slave once the backup finishes importing.
+
+For step #4, add the :ref:`max-commit-id option <slave_cfg_master_options>` to the :ref:`slave_config_file`:
+
+.. code-block:: ini
+
+   # Example of existing lines
+   [master1]
+   master-host=10.0.0.1
+   master-user=slave
+   master-pass=foo
+
+   # Add this line
+   max-commit-id=3303
+
+The value for :ref:`max-commit-id <slave_cfg_master_options>` is the ``COMMIT_ID`` value from the ``SYS_REPLICATION_LOG`` comment in the master dump file (steps #1 and #2).  This value defines the committed transaction ID on the master from which the slave will start applying transactions.
+
+Finally, start the slave *with* the :ref:`slave_plugin` and verify that replication is working (see :ref:`slave_admin`). 

=== added file 'docs/replication/examples/slave/simple_master_slave.rst'
--- docs/replication/examples/slave/simple_master_slave.rst	1970-01-01 00:00:00 +0000
+++ docs/replication/examples/slave/simple_master_slave.rst	2012-03-18 00:36:19 +0000
@@ -0,0 +1,172 @@
+.. program:: drizzled
+
+.. _simple_master_slave_example:
+
+Simple Master-Slave
+===================
+
+:Synopsis: Set up one master and one slave, neither of which have existing data
+:Replicator: :ref:`default_replicator`
+:Applier: :ref:`slave`
+:Version: :ref:`slave_1.1_drizzle_7.1`
+:Authors: Daniel Nichter
+
+This example uses new servers that are not yet configured and do not yet have any data.  If Drizzle was installed from a binary package or repository, then it may already be configured, in which case the following steps may not work or conflict with the existing configuration.  In any case, the principles in this example can be adapted to different configurations.
+
+Master Setup
+------------
+
+First, temporarily start the master server with no :ref:`authentication` in order to create the user accounts.
+
+.. code-block:: bash
+
+   sbin/drizzled >> /var/log/drizzled.log 2>&1 &
+
+Once Drizzle is running, create the user accounts:
+
+.. code-block:: mysql
+
+   drizzle> CREATE SCHEMA auth;
+   
+   drizzle> USE auth;
+   
+   drizzle> CREATE TABLE users (
+         ->   user     VARCHAR(255) NOT NULL,
+         ->   password VARCHAR(40),
+         ->   UNIQUE INDEX user_idx (user)
+         -> );
+
+   drizzle> INSERT INTO auth.users (user, password)
+         -> VALUES ('root', MYSQL_PASSWORD('foo')),
+         ->        ('slave', MYSQL_PASSWORD('foo'));
+
+   drizzle> shutdown;
+
+Once Drizzle has stopped running, restart it with the :ref:`auth_schema_plugin` plugin and other plugins required for :ref:`configuring_a_master`:
+
+.. code-block:: bash
+
+   sbin/drizzled                                \
+      --daemon                                  \
+      --pid-file /var/run/drizzled/drizzled.pid \
+      --plugin-remove auth_all                  \
+      --plugin-add auth_schema                  \
+      --innodb.replication-log                  \
+   >> /var/log/drizzled.log 2>&1
+
+See the options used to start Drizzle for more information:
+
+* :option:`--daemon`
+* :option:`--pid-file`
+* :option:`--plugin-add`
+* :option:`--innodb.replication-log`
+
+Other options and :ref:`plugins` can be used if necessary.
+
+Verify that the master is running and writing :ref:`replication_events` to the :ref:`innodb_transaction_log`:
+
+.. code-block:: mysql
+
+   $ drizzle --user=root --password=foo
+
+   drizzle> SELECT ID FROM DATA_DICTIONARY.SYS_REPLICATION_LOG LIMIT 1;
+   +------+
+   | ID   |
+   +------+
+   |  772 | 
+   +------+
+
+The query should return a row.  This is the table from which slaves read replication events.
+
+The master is now ready to replicate to slaves.
+
+Slave Setup
+-----------
+
+The slave must be on a different sever than the master, else the two servers will have port conflicts.
+
+Since the slave is new and has no data, it will replicate all data from the master, including the :ref:`auth_schema_plugin` table just created.
+
+First, write a :ref:`slave_config_file`:
+
+.. code-block:: ini
+
+   [master1]
+   master-host=10.0.0.1
+   master-user=slave
+   master-pass=foo
+
+The :ref:`master-host option <slave_cfg_master_options>` must be set to the master server's address.  10.0.0.1 is just an example.
+
+Save the config file in :file:`/etc/drizzle/slave.cfg`.
+
+Then start the slave with the plugins required for :ref:`configuring_a_slave`:
+
+.. code-block:: bash
+
+   sbin/drizzled                                 \
+      --daemon                                   \
+      --pid-file /var/run/drizzled/drizzled.pid  \
+      --plugin-add slave                         \
+      --slave.config-file /etc/drizzle/slave.cfg \
+   >> /var/log/drizzled.log 2>&1
+
+Verify that the slave is running and applying replication events from the master:
+
+.. code-block:: mysql
+
+   $ drizzle --user=root --password=foo
+
+   drizzle> SELECT * FROM sys_replication.io_state\G
+   *************************** 1. row ***************************
+   master_id: 1
+      status: RUNNING
+   error_msg: 
+
+   drizzle> SELECT * FROM sys_replication.applier_state\G
+   *************************** 1. row ***************************
+                 master_id: 1
+    last_applied_commit_id: 23
+   originating_server_uuid: 98ECEA09-BA65-489D-9382-F8D15098B1AE
+   originating_commit_id: 23
+                  status: RUNNING
+               error_msg: 
+
+The column values will vary, but the important column is ``status`` :ref:`RUNNING <slave_thread_statuses>` for both tables.
+
+After at least :ref:`applier-thread-sleep <slave_cfg_common_options>` or :ref:`io-thread-sleep <slave_cfg_common_options>` seconds, the :ref:`auth_schema_plugin` table should replicate from the master:
+
+.. code-block:: mysql
+
+   drizzle> SELECT * FROM auth.users;
+   +-------+------------------------------------------+
+   | user  | password                                 |
+   +-------+------------------------------------------+
+   | root  | F3A2A51A9B0F2BE2468926B4132313728C250DBF | 
+   | slave | F3A2A51A9B0F2BE2468926B4132313728C250DBF | 
+   +-------+------------------------------------------+
+
+If not, then check the error log (:file:`/var/log/drizzled.log`) for errors.
+
+Check the :ref:`slave_lag` to ensure that the slave has applied all committed transactions from the master.
+
+Once the slave is fully caught up to the master, stop it, then start it again with the :ref:`auth_schema_plugin` plugin:
+
+.. code-block:: bash
+
+   sbin/drizzled                                 \
+      --daemon                                   \
+      --pid-file /var/run/drizzled/drizzled.pid  \
+      --plugin-add slave                         \
+      --slave.config-file /etc/drizzle/slave.cfg \
+      --plugin-remove auth_all                   \
+      --plugin-add auth_schema                   \
+   >> /var/log/drizzled.log 2>&1
+
+The root username and password should be required:
+
+.. code-block:: bash
+ 
+   $ drizzle --user=root --password=foo
+
+The :ref:`sys_replication_tables` should still show that the :ref:`slave_threads` are :ref:`RUNNING <slave_thread_statuses>`.  Any changes on the master should replicate to the slave within a few seconds.  If any problems occur, consult the :ref:`slave` documentation or ask for :ref:`help`.  Else, congratulations: you are up and running with slave-based Drizzle replication!  Be sure to familiarize yourself with :ref:`slave_admin`.

=== added file 'docs/replication/index.rst'
--- docs/replication/index.rst	1970-01-01 00:00:00 +0000
+++ docs/replication/index.rst	2012-03-18 00:36:19 +0000
@@ -0,0 +1,172 @@
+.. program:: drizzled
+
+.. _drizzle_replication:
+
+Drizzle Replication
+===================
+
+.. topic:: Want to skip the details and setup Drizzle replicaiton as quickly as possible?
+
+   Then start with the
+   :ref:`Simple Master-Slave example <simple_master_slave_example>`.
+   Otherwise, start here and read each section in sequence for a complete
+   understanding of Drizzle replication.
+
+Drizzle encodes replication events as
+`Google Protocol Buffer <http://code.google.com/p/protobuf/>`_ (GPB) messages
+and transmits them asynchronously through replication streams.  A replication
+stream is a pair of one replicator and one applier.  The kernel encodes
+replication events, sends them to all available replicators, which in turn send
+them to their paired appliers.  For Drizzle, the replication process ends at
+this point, but appliers are responsible for applying replication events to a
+service: another Drizzle server (a slave), a message queue, etc.
+
+.. code-block:: none
+
+  Drizzle                                                        External
+  ==========================================================     ========
+  User      Kernel                 Plugins
+  =====     =================      =========================
+                                   Replication Streams
+                                   =========================
+  event --> encode GBP(event) +--> replicator1 +--> applier1 --> service1
+                              |                |
+                              |                +--> applier2 --> service2
+                              |
+                              +--> replicator2 +--> applier3 --> service3
+
+Replicators and appliers are implemented by plugins, so Drizzle replication
+is extensible and varies depending on which plugins are used.  Drizzle
+includes plugins to implement several replication systems:
+
+* :ref:`Master-slave replication, including multi-master <slave_applier>`
+* :ref:`Replication to a RabbitMQ server <rabbitmq_applier>`
+* :ref:`Replication to a ZeroMQ socket <zeromq_applier>`
+
+Master-slave is the most common replication system; it resembles other
+database servers like MySQL. 
+
+Unlike other database servers, the Drizzle kernel plays only a minimal role
+in the replication process: it simply encodes and sends replication events
+to available replicators.  Consequently, there are very few
+:ref:`drizzled replication options <drizzled_replication_options>`, and
+Drizzle replication is primarily configured by replicator and applier
+plugin options.
+
+In summary, Drizzle replication:
+
+* is asynchronous
+* encodes replication events as Google Buffer Protocol messages
+* sends replication events through replication streams (unique replicator-applier pairs)
+* only uses the kernel to encode and send replication events to avaiable replicators
+* is primarily implemented and configured by replicator and applier plugins
+* is extensible
+
+Learned enough?  Ready to start using Drizzle replication?  Then jump to the
+:ref:`replication examples <replication_examples>`.  Otherwise, continue
+reading for more details about all the parts and processes of Drizzle
+replication.
+
+.. _replication_events:
+
+Replication Events
+------------------
+
+Replication events are, in general, any SQL statements which change data or
+schema objects on a server.  :doc:`/dml` queries are the primary cause of
+replication events, but other statements like setting global variables or
+altering schemas or tables may also cause replication events.
+
+The Drizzle kernel sends every replication event to every applier (that is
+to say, every loaded applier plugin), but appliers determine if replicaiton
+events are sent to their paired appliers.  For example, a replicator may
+filter certain replication events.  Likewise, appliers determine if (and how)
+replication events are ultimately applied.
+
+Replication events are not logged or saved by the Drizzle kernel or any
+replicators that ship with Drizzle.  An applier may log replication events
+that it receives from its replicator.
+
+Every replication event is encapsulated as an atomic transaction, including
+bulk and mutli-statement events.
+
+Drizzle relinquishes all control of replication events once they enter a
+replication stream.  Replicators and appliers are responsbile for handling
+replication events correctly and efficiently.
+
+.. _replication_streams:
+
+Replication Streams
+-------------------
+
+Replication stream are logical conduits created by pairing one replicator
+with one applier.  As logical entities, replicaiton streams exist only inside
+the :program:`drizzled` process and cannot be accessed externally.  However,
+some appliers create or access ports or sockets which allows indirect access
+to the replication stream.  Since replicators and appliers are implemented
+by plugins, one could in theory program a custom applier or replicator to
+provide a socket or port for direct access into the replication stream.
+   
+When :program:`drizlzed` starts, it creates replication streams automatically
+based on which replicators are loaded and which appliers are loaded and
+configured to use them.  For example, an applier plugin may be configured
+to use a specific replicator, in which case :program:`drizzled` pairs the
+applier to the specified replicator.  The user does not need to perform
+special steps to create a replication stream.
+
+Replication stream cannot be dynamically recreated; the user must stop
+Drizzle, reconfigure the replicator or applier, and then restart Drizzle to
+let it automatically recreate the new replication stream.
+
+.. _originating_server:
+
+Originating Server
+------------------
+
+The originating server of a replication event is the server on which the
+SQL statement that caused the replication was first executed.  Since one
+replicaiton event may be applied to several services (by passing through
+multiple replication streams), the originating server uniquely identifies
+the true origin of a replication event versus its most immediate upstream
+origin which may have received the replication event from any number of
+additional upstream sources.
+
+Drizzle automatically generates a UUID for every server, saved in the
+:file:`server.uuid` file in the :option:`--datadir` directory.  This UUID
+is included with every replication event that originates from the server.
+
+An originating server may or may not contain both end points of a replication
+stream.  Replicators are always local to (loaded and ran from) the originating
+server from which they receive replication events, but appliers may be local
+or remote (loaded and ran on a different server).  The external service to
+which the applier applies replication events is usually another server,
+not the originating server, but an applier could, in theory, apply events
+from and to the same originating server.
+
+Configuration
+-------------
+
+Drizzle replication is primarily configured by options specific to
+each :ref:`replicator <replicators>` and :ref:`applier <appliers>`.
+
+The Drizzle kernel has very few :ref:`drizzled_replication_options` which
+typically do not need to be changed:
+
+:option:`--transaction-message-threshold`
+    Controls the size, in bytes, of the transaction messages.
+    When a transaction message exceeds this size, a new transaction message
+    with the same transaction ID will be created to continue the replication
+    events.  See :ref:`bulk-operations`.
+
+:option:`--replicate-query`
+    Controls whether the originating SQL query will be included within each
+    statement message contained in the enclosing transaction message. The
+    default global value is FALSE which will not include the query in the
+    messages. It can be controlled per session, as well. For example:
+
+    .. code-block:: mysql
+
+       drizzle> SET @@replicate_query = 1;
+
+    The stored query should be used as a guide only, and never executed
+    on a slave to perform replication as this will lead to incorrect results.

=== added directory 'docs/replication/messages'
=== added file 'docs/replication/messages/index.rst'
--- docs/replication/messages/index.rst	1970-01-01 00:00:00 +0000
+++ docs/replication/messages/index.rst	2012-03-18 00:36:19 +0000
@@ -0,0 +1,251 @@
+Messages
+********
+
+Message Definitions
+===================
+
+The GPB messages are defined in .proto files in the drizzled/message
+directory of the Drizzle source code. The primary definition file is
+transaction.proto. Messages defined in this file are related in the
+following ways::
+
+
+  ------------------------------------------------------------------
+  |                                                                |
+  | Transaction message                                            |
+  |                                                                |
+  |   -----------------------------------------------------------  |
+  |   |                                                         |  |
+  |   | TransactionContext message                              |  |
+  |   |                                                         |  |
+  |   -----------------------------------------------------------  |
+  |   -----------------------------------------------------------  |
+  |   |                                                         |  |
+  |   | Statement message 1                                     |  |
+  |   |                                                         |  |
+  |   -----------------------------------------------------------  |
+  |   -----------------------------------------------------------  |
+  |   |                                                         |  |
+  |   | Statement message 2                                     |  |
+  |   |                                                         |  |
+  |   -----------------------------------------------------------  |
+  |                             ...                                |
+  |   -----------------------------------------------------------  |
+  |   |                                                         |  |
+  |   | Statement message N                                     |  |
+  |   |                                                         |  |
+  |   -----------------------------------------------------------  |
+  ------------------------------------------------------------------
+
+with each Statement message looking like so::
+
+  ------------------------------------------------------------------
+  |                                                                |
+  | Statement message                                              |
+  |                                                                |
+  |   -----------------------------------------------------------  |
+  |   |                                                         |  |
+  |   | Common information                                      |  |
+  |   |                                                         |  |
+  |   |  - Type of Statement (INSERT, DELETE, etc)              |  |
+  |   |  - Start Timestamp                                      |  |
+  |   |  - End Timestamp                                        |  |
+  |   |  - (OPTIONAL) Actual SQL query string                   |  |
+  |   |                                                         |  |
+  |   -----------------------------------------------------------  |
+  |   -----------------------------------------------------------  |
+  |   |                                                         |  |
+  |   | Statement subclass message 1 (see below)                |  |
+  |   |                                                         |  |
+  |   -----------------------------------------------------------  |
+  |                             ...                                |
+  |   -----------------------------------------------------------  |
+  |   |                                                         |  |
+  |   | Statement subclass message N (see below)                |  |
+  |   |                                                         |  |
+  |   -----------------------------------------------------------  |
+  ------------------------------------------------------------------
+
+The Transaction Message
+=======================
+
+The main "envelope" message which represents an atomic transaction
+which changed the state of a server is the Transaction message class.
+
+The Transaction message contains two pieces:
+
+#. A TransactionContext message containing information about the
+   transaction as a whole, such as the ID of the executing server,
+   the start and end timestamp of the transaction, segmenting
+   metadata and a unique identifier for the transaction.
+#. A vector of Statement messages representing the distinct SQL
+   statements which modified the state of the server.  The Statement
+   message is, itself, a generic envelope message containing a
+   sub-message which describes the specific data modification which
+   occurred on the server (such as, for instance, an INSERT statement).
+
+The Statement Message
+=====================
+
+The generic "envelope" message containing information common to each
+SQL statement executed against a server (such as a start and end timestamp
+and the type of the SQL statement) as well as a Statement subclass message
+describing the specific data modification event on the server.
+
+Each Statement message contains a type member which indicates how readers
+of the Statement should construct the inner Statement subclass representing
+a data change.
+
+Statements are recorded separately as sometimes individual statements
+have to be rolled back.
+
+
+.. _bulk-operations:
+
+How Bulk Operations Work
+========================
+
+Certain operations which change large volumes of data on a server
+present a specific set of problems for a transaction coordinator or
+replication service. If all operations must complete atomically on a
+publishing server before replicas are delivered the complete
+transactional unit:
+
+#. The publishing server could consume a large amount of memory
+   building an in-memory Transaction message containing all the
+   operations contained  in the entire transaction.
+#. A replica, or subscribing server, is wasting time waiting on the
+   eventual completion (commit) of the large transaction on the
+   publishing server. It could be applying pieces of the large
+   transaction in the meantime...
+
+In order to prevent the problems inherent in (1) and (2) above, Drizzle's
+replication system uses a mechanism which provides bulk change
+operations.
+
+A single transaction in the database can possibly be represented with
+multiple protobuf Transaction messages if the message grows too large.
+This can happen if you have a bulk transaction, or a single statement
+affecting a very large number of rows, or just a large transaction with
+many statements/changes.
+
+For the first two examples, it is likely that the Statement sub-message
+itself will get segmented, causing another Transaction message to be
+created to hold the rest of the Statement's row changes. In these cases,
+it is enough to look at the segment information stored in the Statement
+message (see example below).
+
+For the last example, the Statement sub-messages may or may not be
+segmented, but we could still need to split the individual Statements up into
+multiple Transaction messages to keep the Transaction message size from
+growing too large. In this case, the segment information in the Statement
+submessages is not helpful if the Statement isn't segmented. We need this
+information in the Transaction message itself.
+
+Segmenting a Single SQL Statement
+=================================
+
+When a regular SQL statement modifies or inserts more rows than a
+certain threshold, Drizzle's replication services component will begin
+sending Transaction messages to replicas which contain a chunk
+(or "segment") of the data which has been changed on the publisher.
+
+When data is inserted, updated, or modified in the database, a
+header containing information about modified tables and fields is
+matched with one or more data segments which contain the actual
+values changed in the statement.
+
+It's easiest to understand this mechanism by following through a real-world
+scenario.
+
+Suppose the following table:
+
+.. code-block:: mysql
+
+  CREATE TABLE test.person
+  (
+    id INT NOT NULL AUTO_INCREMENT PRIMARY KEY
+  , first_name VARCHAR(50)
+  , last_name VARCHAR(50)
+  , is_active CHAR(1) NOT NULL DEFAULT 'Y'
+  );
+
+Also suppose that test.t1 contains 1 million records.
+
+Next, suppose a client issues the SQL statement:
+
+.. code-block:: mysql
+
+  UPDATE test.person SET is_active = 'N';
+
+It is clear that one million records could be updated by this statement
+(we say, "could be" since Drizzle does not actually update a record if
+the UPDATE would not change the existing record...).
+
+In order to prevent the publishing server from having to construct an
+enormous Transaction message, Drizzle's replication services component
+will do the following:
+
+#. Construct a Transaction message with a transaction context containing
+   information about the originating server, the transaction ID, and
+   timestamp information.
+#. Construct an UpdateHeader message with information about the tables
+   and fields involved in the UPDATE statement.  Push this UpdateHeader
+   message onto the Transaction message's statement vector.
+#. Construct an UpdateData message.  Set the *segment_id* member to 1.
+   Set the *end_segment* member to true.
+#. For every record updated in a storage engine, the ReplicationServices
+   component builds a new UpdateRecord message and appends this message
+   to the aforementioned UpdateData message's record vector.
+#. After a certain threshold of records is reached, the
+   ReplicationServices component sets the current UpdateData message's
+   *end_segment* member to false, and proceeds to send the Transaction
+   message to replicators.
+#. The ReplicationServices component then constructs a new Transaction
+   message and constructs a transaction context with the same
+   transaction ID and server information.
+#. A new UpdateData message is created.  The message's *segment_id* is
+   set to N+1 and as new records are updated, new UpdateRecord messages
+   are appended to the UpdateData message's record vector.
+#. While records are being updated, we repeat steps 5 through 7, with
+   only the final UpdateData message having its *end_segment* member set
+   to true.
+
+Segmenting a Transaction
+========================
+
+The Transaction protobuf message also contains *segment_id* member and a
+*end_segment* member. These values are also set appropriately when a
+Statement sub-message is segmented, as described above.
+
+These values are also set when a Transaction must be segmented along
+individual Statement boundaries (i.e., the Statement message itself
+is **not** segmented). In either case, it is enough to check the
+*end_segment* and *segment_id* values of the Transaction message
+to determine if this is a multi-message transaction.
+
+Handling ROLLBACKs
+==================
+
+Both transactions and individual statements may be rolled back.
+
+When a transaction is rolled back, one of two things happen depending
+on whether the transaction is made up of either a single Transaction
+message, or if it is made up of multiple Transaction messages (e.g, bulk
+load).
+
+* For a transaction encapsulated entirely within a single Transaction
+  message, the entire message is simply discarded and not sent through
+  the replication stream.
+* For a transaction which is made up of multiple messages, and at least
+  one message has already been sent through the replication stream, then
+  the Transaction message will contain a Statement message with type =
+  ROLLBACK. This signifies to rollback the entire transaction.
+
+A special Statement message type, ROLLBACK_STATEMENT, is used when
+we have a segmented Statement message (see above) and we need to tell the
+receiver to undo any changes made for this single statement, but not
+for the entire transaction. If the receiver cannot handle rolling back
+a single statement, then a message buffering strategy should be employed 
+to guarantee that a statement was indeed applied successfully before
+executing on the receiver.

=== added directory 'docs/replication/replicators'
=== added file 'docs/replication/replicators/default.rst'
--- docs/replication/replicators/default.rst	1970-01-01 00:00:00 +0000
+++ docs/replication/replicators/default.rst	2012-03-18 00:36:19 +0000
@@ -0,0 +1,64 @@
+.. _default_replicator:
+
+Default Replicator
+==================
+
+The default replicator plugin, cleverly named ``default_replicator``,
+does not modify or filter any replication events; it simply sends every
+replication event it receives from the Drizzle kernel to every applier
+with which it is paired.
+
+:ref:`appliers` default to this replicator.
+
+.. _default_replicator_loading:
+
+Loading
+-------
+
+This plugin is loaded by default.   To stop the plugin from loading by
+default, start :program:`drizzled` with::
+
+   --plugin-remove=default_replicator
+
+.. seealso:: :ref:`drizzled_plugin_options` for more information about adding and removing plugins.
+
+.. _default_replicator_configuration:
+
+Configuration
+-------------
+
+This plugin does not have any command line options.
+
+.. _default_replicator_variables:
+
+Variables
+---------
+
+This plugin does not register any variables.
+
+.. _default_replicator_authors:
+
+Authors
+-------
+
+Jay Pipes
+
+.. _default_replicator_version:
+
+Version
+-------
+
+This documentation applies to **default_replicator 1.0**.
+
+To see which version of the plugin a Drizzle server is running, execute:
+
+.. code-block:: mysql
+
+   SELECT MODULE_VERSION FROM DATA_DICTIONARY.MODULES WHERE MODULE_NAME='default_replicator'
+
+Changelog
+---------
+
+v1.0
+^^^^
+* First release.

=== added file 'docs/replication/replicators/filtered.rst'
--- docs/replication/replicators/filtered.rst	1970-01-01 00:00:00 +0000
+++ docs/replication/replicators/filtered.rst	2012-03-18 00:36:19 +0000
@@ -0,0 +1,143 @@
+.. _filtered_replicator:
+
+Filtered Replicator
+===================
+
+The filtered replicator plugin, named ``filtered_replicator``, filters
+replication events based on schema name or table name.  Regular expressions
+can be used for the schema and table names.
+
+.. _filtered_replicator_loading:
+
+Loading
+-------
+
+To load this plugin, start :program:`drizzled` with::
+
+   --plugin-add=filtered_replicator
+
+Loading the plugin may not enable or configure it.  See the plugin's
+:ref:`filtered_replicator_configuration` and :ref:`filtered_replicator_variables`.
+
+.. seealso:: :ref:`drizzled_plugin_options` for more information about adding and removing plugins.
+
+.. _filtered_replicator_configuration:
+
+Configuration
+-------------
+
+These command line options configure the plugin when :program:`drizzled`
+is started.  See :ref:`command_line_options` for more information about
+specifying command line options.
+
+.. program:: drizzled
+
+.. option:: --filtered-replicator.filteredschemas ARG
+
+   :Default: 
+   :Variable: :ref:`filtered_replicator_filteredschemas <filtered_replicator_filteredschemas>`
+
+   Comma-separated list of schemas to exclude from replication.
+
+.. option:: --filtered-replicator.filteredtables ARG
+
+   :Default: 
+   :Variable: :ref:`filtered_replicator_filteredtables <filtered_replicator_filteredtables>`
+
+   Comma-separated list of tables to exclude from replication.
+
+.. option:: --filtered-replicator.schemaregex ARG
+
+   :Default: 
+   :Variable: :ref:`filtered_replicator_schemaregex <filtered_replicator_schemaregex>`
+
+   Regular expression to apply to schemas to exclude from replication.
+
+.. option:: --filtered-replicator.tableregex ARG
+
+   :Default: 
+   :Variable: :ref:`filtered_replicator_tableregex <filtered_replicator_tableregex>`
+
+   Regular expression to apply to tables to exclude from replication.
+
+.. _filtered_replicator_variables:
+
+Variables
+---------
+
+These variables show the running configuration of the plugin.
+See `variables` for more information about querying and setting variables.
+
+.. _filtered_replicator_filteredschemas:
+
+* ``filtered_replicator_filteredschemas``
+
+   :Scope: Global
+   :Dynamic: No
+   :Option: :option:`--filtered-replicator.filteredschemas`
+
+   Comma-separated list of schemas to exclude from replication.
+
+.. _filtered_replicator_filteredtables:
+
+* ``filtered_replicator_filteredtables``
+
+   :Scope: Global
+   :Dynamic: No
+   :Option: :option:`--filtered-replicator.filteredtables`
+
+   Comma-separated list of tables to exclude from replication.
+
+.. _filtered_replicator_schemaregex:
+
+* ``filtered_replicator_schemaregex``
+
+   :Scope: Global
+   :Dynamic: No
+   :Option: :option:`--filtered-replicator.schemaregex`
+
+   Regular expression to apply to schemas to exclude from replication.
+
+.. _filtered_replicator_tableregex:
+
+* ``filtered_replicator_tableregex``
+
+   :Scope: Global
+   :Dynamic: No
+   :Option: :option:`--filtered-replicator.tableregex`
+
+   Regular expression to apply to tables to exclude from replication.
+
+.. _filtered_replicator_examples:
+
+Examples
+--------
+
+Sorry, there are no examples for this plugin.
+
+.. _filtered_replicator_authors:
+
+Authors
+-------
+
+Padraig O Sullivan
+
+.. _filtered_replicator_version:
+
+Version
+-------
+
+This documentation applies to **filtered_replicator 0.2**.
+
+To see which version of the plugin a Drizzle server is running, execute:
+
+.. code-block:: mysql
+
+   SELECT MODULE_VERSION FROM DATA_DICTIONARY.MODULES WHERE MODULE_NAME='filtered_replicator'
+
+Chagnelog
+---------
+
+v0.2
+^^^^
+* First release.

=== added file 'docs/replication/replicators/index.rst'
--- docs/replication/replicators/index.rst	1970-01-01 00:00:00 +0000
+++ docs/replication/replicators/index.rst	2012-03-18 00:36:19 +0000
@@ -0,0 +1,28 @@
+.. _replicators:
+
+.. _replication_replicators:
+
+Replicators
+===========
+
+Replicators are one end point of a
+:ref:`replication stream <replication_streams>` which provide an interface
+between the Drizzle kernel and :ref:`appliers`.  The Drizzle kernel sends
+replication events, encoded as Google Buffer Protocol message, to every
+replicator that was loaded when Drizzled was stared.
+Replicators implement specific behaviors by modifying or filtering
+replication events before sending them to their paired appliers.
+For example, a replicator may filter replication events from specific schemas
+or tables.  There are no restrictions on how a replicator can modify
+replication events.
+
+Replicators are implemented by plugins and have unique names.  Most appliers
+can be configured to use a specific replicator by specifying its unique name.
+
+Drizzle includes the following replicator plugins:
+
+.. toctree::
+   :maxdepth: 1
+   :glob:
+
+   *

=== modified file 'plugin/default_replicator/docs/index.rst'
--- plugin/default_replicator/docs/index.rst	2011-10-23 05:45:09 +0000
+++ plugin/default_replicator/docs/index.rst	2012-03-18 00:36:19 +0000
@@ -1,66 +1,4 @@
-.. _default_replicator_plugin:
-
 Default Replicator
 ==================
 
-:program:`default_replicator` is a simple replicator which replicates all
-write events to all appliers.
-
-.. seealso:: :doc:`/replication/drizzle`
-
-.. _default_replicator_loading:
-
-Loading
--------
-
-This plugin is loaded by default, but it may need to be configured.  See
-the plugin's :ref:`default_replicator_configuration` and
-:ref:`default_replicator_variables`.
-
-To stop the plugin from loading by default, start :program:`drizzled`
-with::
-
-   --plugin-remove=default_replicator
-
-.. seealso:: :ref:`drizzled_plugin_options` for more information about adding and removing plugins.
-
-.. _default_replicator_configuration:
-
-Configuration
--------------
-
-This plugin does not have any command line options.
-
-.. _default_replicator_variables:
-
-Variables
----------
-
-This plugin does not register any variables.
-
-.. _default_replicator_authors:
-
-Authors
--------
-
-Jay Pipes
-
-.. _default_replicator_version:
-
-Version
--------
-
-This documentation applies to **default_replicator 1.0**.
-
-To see which version of the plugin a Drizzle server is running, execute:
-
-.. code-block:: mysql
-
-   SELECT MODULE_VERSION FROM DATA_DICTIONARY.MODULES WHERE MODULE_NAME='default_replicator'
-
-Changelog
----------
-
-v1.0
-^^^^
-* First release.
+See :ref:`default_replicator`.

=== modified file 'plugin/filtered_replicator/docs/index.rst'
--- plugin/filtered_replicator/docs/index.rst	2011-10-23 05:45:09 +0000
+++ plugin/filtered_replicator/docs/index.rst	2012-03-18 00:36:19 +0000
@@ -1,145 +1,4 @@
-.. _filtered_replicator_plugin:
-
 Filtered Replicator
 ===================
 
-The Filtered Replicator plugin registers itself in the Drizzle kernel
-replication stream process as a new replicator (see :ref:`replication_streams`
-for more information). It provides a way to filter replication messages by
-schema name or table name.  Regular expressions can be used for the schema and
-table names.
-
-.. _filtered_replicator_loading:
-
-Loading
--------
-
-To load this plugin, start :program:`drizzled` with::
-
-   --plugin-add=filtered_replicator
-
-Loading the plugin may not enable or configure it.  See the plugin's
-:ref:`filtered_replicator_configuration` and :ref:`filtered_replicator_variables`.
-
-.. seealso:: :ref:`drizzled_plugin_options` for more information about adding and removing plugins.
-
-.. _filtered_replicator_configuration:
-
-Configuration
--------------
-
-These command line options configure the plugin when :program:`drizzled`
-is started.  See :ref:`command_line_options` for more information about specifying
-command line options.
-
-.. program:: drizzled
-
-.. option:: --filtered-replicator.filteredschemas ARG
-
-   :Default: 
-   :Variable: :ref:`filtered_replicator_filteredschemas <filtered_replicator_filteredschemas>`
-
-   Comma-separated list of schemas to exclude from replication.
-
-.. option:: --filtered-replicator.filteredtables ARG
-
-   :Default: 
-   :Variable: :ref:`filtered_replicator_filteredtables <filtered_replicator_filteredtables>`
-
-   Comma-separated list of tables to exclude from replication.
-
-.. option:: --filtered-replicator.schemaregex ARG
-
-   :Default: 
-   :Variable: :ref:`filtered_replicator_schemaregex <filtered_replicator_schemaregex>`
-
-   Regular expression to apply to schemas to exclude from replication.
-
-.. option:: --filtered-replicator.tableregex ARG
-
-   :Default: 
-   :Variable: :ref:`filtered_replicator_tableregex <filtered_replicator_tableregex>`
-
-   Regular expression to apply to tables to exclude from replication.
-
-.. _filtered_replicator_variables:
-
-Variables
----------
-
-These variables show the running configuration of the plugin.
-See `variables` for more information about querying and setting variables.
-
-.. _filtered_replicator_filteredschemas:
-
-* ``filtered_replicator_filteredschemas``
-
-   :Scope: Global
-   :Dynamic: No
-   :Option: :option:`--filtered-replicator.filteredschemas`
-
-   Comma-separated list of schemas to exclude from replication.
-
-.. _filtered_replicator_filteredtables:
-
-* ``filtered_replicator_filteredtables``
-
-   :Scope: Global
-   :Dynamic: No
-   :Option: :option:`--filtered-replicator.filteredtables`
-
-   Comma-separated list of tables to exclude from replication.
-
-.. _filtered_replicator_schemaregex:
-
-* ``filtered_replicator_schemaregex``
-
-   :Scope: Global
-   :Dynamic: No
-   :Option: :option:`--filtered-replicator.schemaregex`
-
-   Regular expression to apply to schemas to exclude from replication.
-
-.. _filtered_replicator_tableregex:
-
-* ``filtered_replicator_tableregex``
-
-   :Scope: Global
-   :Dynamic: No
-   :Option: :option:`--filtered-replicator.tableregex`
-
-   Regular expression to apply to tables to exclude from replication.
-
-.. _filtered_replicator_examples:
-
-Examples
---------
-
-Sorry, there are no examples for this plugin.
-
-.. _filtered_replicator_authors:
-
-Authors
--------
-
-Padraig O Sullivan
-
-.. _filtered_replicator_version:
-
-Version
--------
-
-This documentation applies to **filtered_replicator 0.2**.
-
-To see which version of the plugin a Drizzle server is running, execute:
-
-.. code-block:: mysql
-
-   SELECT MODULE_VERSION FROM DATA_DICTIONARY.MODULES WHERE MODULE_NAME='filtered_replicator'
-
-Chagnelog
----------
-
-v0.2
-^^^^
-* First release.
+See :ref:`filtered_replicator`.

=== modified file 'plugin/innobase/docs/index.rst'
--- plugin/innobase/docs/index.rst	2012-03-08 00:54:48 +0000
+++ plugin/innobase/docs/index.rst	2012-03-18 00:36:19 +0000
@@ -49,10 +49,10 @@
 
 .. _innodb_transaction_log:
 
-InnoDB Transaction Log
+InnoDB Replicaiton Log
 ----------------------
 
-The :program:`innodb` plugin provides a mechanism to store replication
+The ``innodb`` plugin provides a mechanism to store replication
 events in an InnoDB table. When enabled, this transaction log can be accessed
 through the SYS_REPLICATION_LOG and INNODB_REPLICATION_LOG tables in the
 DATA_DICTIONARY schema.
@@ -524,6 +524,13 @@
 
    Use InnoDB's internal memory allocator instead of the system's malloc.
 
+.. option:: --innodb.use-replicator
+
+   :Default: default
+   :Variable: `innodb_use_replicator <innodb_use_sys_malloc>`
+
+   Use this replicator for the :ref:`innodb_transaction_log`.
+
 .. option:: --innodb.version ARG
 
    :Default:
@@ -1110,6 +1117,16 @@
 
    If system or internal malloc() is being used.
 
+.. _innodb_use_replicator:
+
+* ``innodb_use_replicator``
+
+   :Scope: Global
+   :Dynamic: No
+   :Option: :option:`--innodb.use-replicator`
+
+   Replicator to which the :ref:`innodb_transaction_log` is paired.
+
 .. _innodb_version_var:
 
 * ``innodb_version``

=== modified file 'plugin/rabbitmq/docs/index.rst'
--- plugin/rabbitmq/docs/index.rst	2011-10-23 05:45:09 +0000
+++ plugin/rabbitmq/docs/index.rst	2012-03-18 00:36:19 +0000
@@ -1,243 +1,6 @@
 .. _rabbitmq_plugin:
 
 RabbitMQ Integration
-======================
-
-It is possible to replicate transactions directly to a `RabbitMQ <http://www.rabbitmq.org>`_ server from drizzle, this could be used to create advanced replication solutions, to visualize data, or to build triggers. For example, `RabbitReplication <http://www.rabbitreplication.org>`_ has been built to consume the messages from rabbitmq, transform them, and persist the data in drizzle, mysql, many nosql stores and even replicating directly to websockets for data visualization.
-
-Loading
--------
-
-To load this plugin, start :program:`drizzled` with::
-
-   --plugin-add=rabbitmq
-
-Loading the plugin may not enable or configure it.  See the plugin's
-:ref:`rabbitmq_configuration` and :ref:`rabbitmq_variables`.
-
-.. seealso:: :ref:`drizzled_plugin_options` for more information about adding and removing plugins.
-
-.. _rabbitmq_configuration:
-
-Configuration
--------------
-
-These command line options configure the plugin when :program:`drizzled`
-is started.  See :ref:`command_line_options` for more information about specifying
-command line options.
-
-.. program:: drizzled
-
-.. option:: --rabbitmq.exchange ARG
-
-   :Default: ReplicationExchange
-   :Variable: :ref:`rabbitmq_exchange <rabbitmq_exchange>`
-
-   Name of RabbitMQ exchange to publish to
-
-.. option:: --rabbitmq.host ARG
-
-   :Default: localhost
-   :Variable: :ref:`rabbitmq_host <rabbitmq_host>`
-
-   Host name to connect to
-
-.. option:: --rabbitmq.password ARG
-
-   :Default: guest
-   :Variable: :ref:`rabbitmq_password <rabbitmq_password>`
-
-   RabbitMQ password
-
-.. option:: --rabbitmq.port ARG
-
-   :Default: 5672
-   :Variable: :ref:`rabbitmq_port <rabbitmq_port>`
-
-   Port to connect to
-
-.. option:: --rabbitmq.routingkey ARG
-
-   :Default: ReplicationRoutingKey
-   :Variable: :ref:`rabbitmq_routingkey <rabbitmq_routingkey>`
-
-   Name of RabbitMQ routing key to use
-
-.. option:: --rabbitmq.use-replicator ARG
-
-   :Default: default_replicator
-   :Variable:
-
-   Name of the replicator plugin to use (default='default_replicator')
-
-.. option:: --rabbitmq.username ARG
-
-   :Default: guest
-   :Variable: :ref:`rabbitmq_username <rabbitmq_username>`
-
-   RabbitMQ username
-
-.. option:: --rabbitmq.virtualhost ARG
-
-   :Default: /
-   :Variable: :ref:`rabbitmq_virtualhost <rabbitmq_virtualhost>`
-
-   RabbitMQ virtualhost
-
-.. _rabbitmq_variables:
-
-Variables
----------
-
-These variables show the running configuration of the plugin.
-See `variables` for more information about querying and setting variables.
-
-.. _rabbitmq_exchange:
-
-* ``rabbitmq_exchange``
-
-   :Scope: Global
-   :Dynamic: No
-   :Option: :option:`--rabbitmq.exchange`
-
-   Name of RabbitMQ exchange to publish to
-
-.. _rabbitmq_host:
-
-* ``rabbitmq_host``
-
-   :Scope: Global
-   :Dynamic: No
-   :Option: :option:`--rabbitmq.host`
-
-   Host name to connect to
-
-.. _rabbitmq_password:
-
-* ``rabbitmq_password``
-
-   :Scope: Global
-   :Dynamic: No
-   :Option: :option:`--rabbitmq.password`
-
-   RabbitMQ password
-
-.. _rabbitmq_port:
-
-* ``rabbitmq_port``
-
-   :Scope: Global
-   :Dynamic: No
-   :Option: :option:`--rabbitmq.port`
-
-   Port to connect to
-
-.. _rabbitmq_routingkey:
-
-* ``rabbitmq_routingkey``
-
-   :Scope: Global
-   :Dynamic: No
-   :Option: :option:`--rabbitmq.routingkey`
-
-   Name of RabbitMQ routing key to use
-
-.. _rabbitmq_username:
-
-* ``rabbitmq_username``
-
-   :Scope: Global
-   :Dynamic: No
-   :Option: :option:`--rabbitmq.username`
-
-   RabbitMQ username
-
-.. _rabbitmq_virtualhost:
-
-* ``rabbitmq_virtualhost``
-
-   :Scope: Global
-   :Dynamic: No
-   :Option: :option:`--rabbitmq.virtualhost`
-
-   RabbitMQ virtualhost
-
-.. _rabbitmq_examples:
-
-Examples
---------
-
-First install a recent version of RabbitMQ, then install librabbitmq, the C library for talking to the RabbitMQ server:
-
-.. code-block:: bash
-
-   $ hg clone http://hg.rabbitmq.com/rabbitmq-codegen/
-   $ hg clone http://hg.rabbitmq.com/rabbitmq-c/
-   $ cd rabbitmq-c
-   $ autoreconf -f -i
-   $ ./configure
-   $ make
-   $ make install
-
-Now you probably need to rebuild Drizzle since the :program:`rabbitmq` plugin is not built if librabbitmq is not installed.
-
-Finally, start :program:`drizzled` like:
-
-.. code-block:: bash
-
-   sbin/drizzled --plugin-add rabbitmq,default-replicator \
-                 --rabbitmq.use-replicator default
-
-To verify that it works, you can start a generic rabbitmq listener from librabbitmq:
-
-.. code-block:: bash
-
-   $ amqp_listen localhost 5672 ReplicationExchange ReplicationRoutingKey
-
-And you should see something like this when you do an INSERT/CREATE/.. (just not a select) in your newly built Drizzle instance::
-
-   Result 0
-   Frame type 1, channel 1
-   Method AMQP_BASIC_DELIVER_METHOD
-   Delivery 1, exchange ReplicationExchange routingkey ReplicationRoutingKey
-
-   00000000: 0A 17 08 01 10 87 36 18 : F0 FA D9 99 FA F1 A7 02  ......6.........
-   00000010: 20 99 81 DA 99 FA F1 A7 : 02 12 40 08 01 10 F2 FA   .........@.....
-   00000020: D9 99 FA F1 A7 02 18 FC : FA D9 99 FA F1 A7 02 2A  ...............*
-   00000030: 17 0A 06 0A 01 62 12 01 : 61 12 06 08 04 12 02 69  .....b..a......i
-   00000040: 64 12 05 08 01 12 01 74 : 32 11 08 01 10 01 1A 0B  d......t2.......
-   00000050: 0A 01 32 0A 02 61 61 10 : 00 10 00 20 01 28 01     ..2..aa.... .(.
-   0000005F:
-
-Implementation Details
-----------------------
-
-* :program:`drizzled` will not sart if the rabbitmq server is not available.
-* If the rabbitmq server goes away, the plugin will try to reconnect and resend the message 3 times, after that, the transaction is rolled back.
-
-.. _rabbitmq_authors:
-
-Authors
--------
-
-Marcus Eriksson
-
-.. _rabbitmq_version:
-
-Version
--------
-
-This documentation applies to **rabbitmq 0.1**.
-
-To see which version of the plugin a Drizzle server is running, execute:
-
-.. code-block:: mysql
-
-   SELECT MODULE_VERSION FROM DATA_DICTIONARY.MODULES WHERE MODULE_NAME='rabbitmq'
-
-Changelog
----------
-
-v0.1
-^^^^
-* First release.
+====================
+
+See :ref:`rabbitmq_applier`.

=== removed file 'plugin/slave/docs/admin.rst'
--- plugin/slave/docs/admin.rst	2011-03-11 18:38:09 +0000
+++ plugin/slave/docs/admin.rst	1970-01-01 00:00:00 +0000
@@ -1,136 +0,0 @@
-********************************
-Replication Slave Administration
-********************************
-
-This page walks you through some common administration tasks when using
-the replication slave plugin.
-
-Monitoring the Master
-#####################
-
-Slave Connections
-*****************
-
-If you want to determine which slave machines are connected to your
-master, use the *SHOW PROCESSLIST* command. Slave connections will show
-up in the output of this command.
-
-InnoDB Transaction Log
-**********************
-
-The slave plugin uses the InnoDB transaction log (see
-:ref:`innodb_transaction_log`) on the master to retrieve replication
-messages. This transaction log, though stored as an internal table within
-InnoDB, offers two different views to the table contents. Two tables in
-the DATA_DICTIONARY schema provide the different views into the transaction
-log: the SYS_REPLICATION_LOG table and the INNODB_REPLICATION_LOG table.
-
-The SYS_REPLICATION_LOG table is read directly by the slave plugin.
-This table is described as below::
-
-  drizzle> SHOW CREATE TABLE data_dictionary.sys_replication_log\G
-  *************************** 1. row ***************************
-         Table: SYS_REPLICATION_LOG
-  Create Table: CREATE TABLE `SYS_REPLICATION_LOG` (
-    `ID` BIGINT,
-    `SEGID` INT,
-    `COMMIT_ID` BIGINT,
-    `END_TIMESTAMP` BIGINT,
-    `MESSAGE_LEN` INT,
-    `MESSAGE` BLOB,
-    PRIMARY KEY (`ID`,`SEGID`) USING BTREE,
-    KEY `COMMIT_IDX` (`COMMIT_ID`,`ID`) USING BTREE
-  ) ENGINE=InnoDB COLLATE = binary
-
-The INNODB_REPLICATION_LOG is similar to the SYS_REPLICATION_LOG, the
-main difference being that the Google Protobuffer message representing
-the changed rows is converted to plain text before being output::
-
-  drizzle> SHOW CREATE TABLE data_dictionary.innodb_replication_log\G
-  *************************** 1. row ***************************
-         Table: INNODB_REPLICATION_LOG
-  Create Table: CREATE TABLE `INNODB_REPLICATION_LOG` (
-    `TRANSACTION_ID` BIGINT NOT NULL,
-    `TRANSACTION_SEGMENT_ID` BIGINT NOT NULL,
-    `COMMIT_ID` BIGINT NOT NULL,
-    `END_TIMESTAMP` BIGINT NOT NULL,
-    `TRANSACTION_MESSAGE_STRING` TEXT COLLATE utf8_general_ci NOT NULL,
-    `TRANSACTION_LENGTH` BIGINT NOT NULL
-  ) ENGINE=FunctionEngine COLLATE = utf8_general_ci REPLICATE = FALSE
-
-The INNODB_REPLICATION_LOG table is read-only due to the way it is
-implemented. The SYS_REPLICATION_LOG table, on the other hand, allows you
-to modify the contents of the transaction log. You would use this table
-to trim the transaction log.
-
-Monitoring the Slave
-####################
-
-The slave plugin has two types of threads doing all of the work:
-
-* An IO (or producer) thread
-* An applier (or consumer) thread
-
-The status of each thread is stored in tables in the *sys_replication*
-schema. The IO thread status is contained in the *io_state* table, and
-the applier thread status is in the *applier_state* table. You may query
-these tables just like any other table. For example::
-
-  drizzle> SELECT * FROM sys_replication.io_state\G
-  *************************** 1. row ***************************
-     status: RUNNING
-  error_msg: 
-
-The above shows that the IO thread is **RUNNING**. If there had been
-an error on the IO thread, the status value would be **STOPPED** and
-the error_msg column would contain information about the error.
-
-We can check the state of the applier thread in a similar manner::
-
-  drizzle> SELECT * FROM sys_replication.applier_state\G
-  *************************** 1. row ***************************
-  last_applied_commit_id: 4
-                  status: RUNNING
-               error_msg: 
-
-The status and error_msg columns are similar to the ones in the *io_state*
-table. Also available is the last_applied_commit_id, which contains the
-value of the COMMIT_ID from the master's replication log (see definition
-of the data_dictionary.sys_replication_log table above) of the most
-recently executed transaction.
-
-Transaction Log Maintenance
-###########################
-
-Currently, the InnoDB transaction log grows without bounds and is never
-trimmed of unneeded entries. This can present a problem for long running
-replication setups. You may trim the log manually, but you must make certain
-to not remove any entries that are needed by slave servers.
-
-Follow these steps to trim the InnoDB transaction without affecting slave
-function:
-
-#. Query each slave for the *last_applied_commit_id* value from the *sys_replication.applier_state* table.
-#. Choose the **minimum** value obtained from step one. This will be the marker for the slave that is the furthest behind the master.
-#. Using this marker value from the previous step, delete all entries from the master's transaction log that has a COMMIT_ID less than the marker value.
-
-Below is an example of the steps defined above. First, step 1 and 2. Assume
-that we have two slave hosts connected to the master (slave-1 and slave-2).
-We need to query both to check their relationship with the master transaction
-log::
-
-  slave-1> SELECT last_applied_commit_id FROM sys_replication.applier_state\G
-  *************************** 1. row ***************************
-  last_applied_commit_id: 3000
-
-  slave-2> SELECT last_applied_commit_id FROM sys_replication.applier_state\G
-  *************************** 1. row ***************************
-  last_applied_commit_id: 2877
-
-We see that slave-2 has the smallest value for *last_applied_commit_id*. We
-will use this value in the next step to trim the transaction log on the
-master::
-
-  master> DELETE FROM data_dictionary.sys_replication_log WHERE commit_id < 2877;
-
-This will remove all old, unneeded entries from the InnoDB transaction log. Note that the SYS_REPLICATION_LOG table is used for this maintenance task.

=== modified file 'plugin/slave/docs/index.rst'
--- plugin/slave/docs/index.rst	2012-03-08 00:54:48 +0000
+++ plugin/slave/docs/index.rst	2012-03-18 00:36:19 +0000
@@ -1,165 +1,4 @@
-.. _slave_plugin:
-
 Replication Slave
 ==================
 
-This page contains the configuration and implementation details for Drizzle replication.
-
-See these pages for user-level information: 
-
-.. toctree::
-   :maxdepth: 2
-
-   user_example
-   admin
-
-Description
------------
-
-Replication enables data from one Drizzle database server (the master) to be replicated to one or more Drizzle database servers (the slaves). It provides a native implementation of replication between two Drizzle processes. Depending on your configuration, you can replicate all databases, selected databases, or selected tables within a database.
-
-Drizzle’s replication system is entirely new and different from MySQL. Replication events are stored using Google Protocol Buffer messages in an InnoDB table. These events are read by the slave, stored locally, and applied. The advantage of the Google Protocol Buffer messages is a script or program can be put together in pretty much any language in minutes, and read the replication log. In other words, replication plugins are easy to implement, which enable developers to entirely customize their replication system.
-
-This plugin requires a master that is running with the InnoDB replication log enabled. The slave plugin allows a server to replicate from another server that is using the innodb-trx log. It is a very simple setup:

-master options:  –innodb.replication-log=true
slave options: –plugin-add=slave –slave.config-file=XYZ
-
-The config file may contain the following options in option=value format:

-master-host – hostname/ip of the master host
master-port – port used by the master server
master-user – username
master-pass – password
max-reconnects – max # of reconnect attempts if the master disappears
seconds-between-reconnects – how long to wait between reconnect attempts
-
-
-Configuration
--------------
-
-Most of the options that can be used to control the replication slave plugin
-can only be given in a configuration file. The only exceptions are the
-:option:`drizzled --slave.config-file` and :option:`drizzled --slave.max-commit-id` options.
-
-.. program:: drizzled
-
-.. option:: --slave.config-file FILE
-
-   :Default: :file:`etc/slave.cfg`
-   :Variable:
-
-   Path to the replication slave configuration file. By default, the
-   plugin will look for a file named :file:`slave.cfg` in the `etc` directory
-   of the Drizzle installation. If you want to specify a different path or
-   configuration file name, it is best to specify a full path to the
-   file. The relative path used by plugins is within the :option:`--datadir`
-   directory, so a full path is recommended.
-
-.. option:: --slave.max-commit-id ID
-
-   Manually set the maximum commit ID the slave is assumed to have retrieved
-   from the master. This value will be used by the slave to determine where
-   to begin retrieving replication messages from the master transaction log.
-   This option can be used to provision a new slave machine by setting it to
-   the value output from the drizzledump client when used with the
-   --single-transaction option.
-
-   This value is not allowed to be set via the configuration file since
-   you would normally only set it once on initial slave startup. This
-   eliminates the possibility of forgetting to delete it from the configuration
-   file for subsequent slave restarts.
-
-The options below are read from the configuration file.
-
-.. confval:: master-host
-
-   Hostname/IP address of the master server.
-
-.. confval:: master-port
-
-   Drizzle port used by the master server. Default is 3306.
-
-.. confval:: master-user
-
-   Username to use for connecting to the master server.
-
-.. confval:: master-pass
-
-   Password associated with the username given by :confval:`master-user`.
-
-.. confval:: max-reconnects
-
-   The number of reconnection attempts the slave plugin will try if the
-   master server becomes unreachable. Default is 10.
-
-.. confval:: seconds-between-reconnects
-
-   The number of seconds to wait between reconnect attempts when the master
-   server becomes unreachable. Default is 30.
-
-.. confval:: io-thread-sleep
-
-   The number of seconds the IO (producer) thread sleeps between queries to the
-   master for more replication events. Default is 5.
-
-.. confval:: applier-thread-sleep
-
-   The number of seconds the applier (consumer) thread sleeps between applying
-   replication events from the local queue. Default is 5.
-
-Implementation Details
-----------------------
-
-The replication slave plugin creates two worker threads, each accessing a
-work queue (implemented as an InnoDB table) that contains the replication
-events. This is a producer/consumer paradigm where one thread populates the
-queue (the producer), and the other thread (the consumer) reads events from
-the queue.
-
-The producer thread (or I/O thread) is in charge of connecting to the master
-server and pulling down replication events from the master's transaction
-log and storing them locally in the slave queue. It is required that the
-master use the InnoDB replication log (:option:`--innodb.replication-log <drizzled --innodb.replication-log>`).
-
-The consumer thread (or applier thread) reads the replication events from
-the local slave queue, applies them locally, and then deletes successfully
-applied events from the queue.
-
-Schemas and Tables
-------------------
-
-The slave plugin creates its own schema and set of tables to store its
-metadata. It stores everything in the **sys_replication** schema. The
-following are the tables that it will create:
-
-.. dbtable:: sys_replication.io_state
-
-   Stores metadata about the IO/producer thread.
-
-.. dbtable:: sys_replication.applier_state
-
-   Stores metadata about the applier/consumer thread.
-
-.. dbtable:: sys_replication.queue
-
-   The replication event queue.
-
-.. _slave_authors:
-
-Authors
--------
-
-David Shrewsbury
-
-.. _slave_version:
-
-Version
--------
-
-This documentation applies to **slave 1.0**.
-
-To see which version of the plugin a Drizzle server is running, execute:
-
-.. code-block:: mysql
-
-   SELECT MODULE_VERSION FROM DATA_DICTIONARY.MODULES WHERE MODULE_NAME='slave'
-
-Changelog
----------
-
-v1.0
-^^^^
-* First release.
+See :ref:`slave_applier`.

=== removed file 'plugin/slave/docs/user_example.rst'
--- plugin/slave/docs/user_example.rst	2011-07-22 23:56:42 +0000
+++ plugin/slave/docs/user_example.rst	1970-01-01 00:00:00 +0000
@@ -1,134 +0,0 @@
-How to use replication: An example
-====================================
-
-A simple replication setup (using a single master and a single slave) between two Drizzle servers is done with the replication slave plugin. With Drizzle replication, you can also provision a new slave into an existing setup.
-
-Replication setup begins with making certain that both master and slave share the same version of Drizzle to avoid any potential incompatibility issues.
-
-Master Setup
--------------
-
-Setting up the master is the first step. An important requirement is to start the master Drizzle database server with the --innodb.replication-log option, and a few other options in most circumstances. More options can be found in the options documentation. These are the most common options needed for a replication master. For example:
-
-    master> usr/local/sbin/drizzled \
-			--innodb.replication-log \
-			--pid-file=/var/run/drizzled/drizzled.pid \
-			--drizzle-protocol.bind-address=0.0.0.0 \
-			--mysql-protocol.bind-address=0.0.0.0 \
-			--daemon
-
-
-Several options are required on most setups. They are set on Drizzle Startup with a --optionname. The most important ones are:
-
-
-The InnoDB replication log must be running:
-
---innodb.replication-log
-
-
-PID must be set:
-
---pid-file=/var/run/drizzled/drizzled.pid
-
-
-the address binding for Drizzle's default port (4427):
-
---drizzle-protocol.bind-address=0.0.0.0
-
-
-The address binding for systems replicating through MySQL's default port (3306):
-
---mysql-protocol.bind-address=0.0.0.0
-
-
-Data Directory can be set other than default:
-
---datadir=$PWD/var
-
-
-For more complex setups, the server id option may be appropriate to use:
-
---server-id
-
-
-To run Drizzle in the background, thereby keeping the database running if the user logs out:
-
---daemon
-
-
-
-With the master running, you can optionally now create a backup of any databases to be imported on the new slave by using :doc:`../../clients/drizzledump`. This example, however, assumes that we are starting with a fresh database with no data.
-
-Slave Setup
--------------
-
-Starting the slave is very similar to starting the master. There are two Drizzle database server options required for the slave: --plugin-add=slave and --slave.config-file. For example: ::
-
- 	slave> /usr/local/sbin/drizzled \
-                        --plugin-add=slave \
-                        --slave.config-file=/usr/local/etc//slave.cfg  
-
-A more typical startup will need more options:
-
-	slave> /usr/local/sbin/drizzled \
-			--plugin-add=slave \
-			--slave.config-file=/usr/local/etc//slave.cfg \
-			--pid-file=/var/run/drizzled/drizzled.pid \
-                        --drizzle-protocol.bind-address=0.0.0.0 \
-                        --mysql-protocol.bind-address=0.0.0.0 \
-                        --daemon
-
-Similar to the Master setup, there are a number of options that can be selected. Please see the Plugin Documentation for the relevent options.
-
-These options tell the server to load the slave plugin, and then tell the slave plugin where to find the slave host configuration file. This configuration file has options to specify the master host and a few options to control how the slave operates. You can read more about the available configuration options in the replication slave plugin documentation. Below is a simple example: ::
-
-	master-host = master.location.com
-	master-port = 4427
-
-Some options that can be set other than default, but are otherwise not necessary, are: 
-
-	master-user = dino_slave
-	master-pass = my_password
-	io-thread-sleep = 10
-	applier-thread-sleep = 10
-
-The slave will immediately connect to the master host specified in the configuration file and begin pulling events from the InnoDB-based transaction log. By default, a freshly provisioned slave will begin pulling from the beginning of this transaction log. Once all replication messages have been pulled from the master and stored locally on the slave host, the IO thread will sleep and periodically awaken to check for more messages. This is straightforward for an initial replication setup. See below to learn about inserting another slave host into an already existing replication architecture.
-
-Provisioning a New Slave Host
--------------------------------
-
-The basic formula for creating a new slave host for an existing replication setup is:
-
-   1. Make a backup of the master databases.
-   2. Record the state of the master transaction log at the point the backup was made.
-   3. Restore the backup on the new slave machine.
-   4. Start the new slave and tell it to begin reading the transaction log from the point recorded in #2.
-
-Steps #1 and #2 are covered with the drizzledump client program. If you use the --single-transaction option to drizzledump, it will place a comment near the beginning of the dump output with the InnoDB transaction log metadata. For example: ::
-
-	master> drizzledump --all-databases --single-transaction > master.backup
-	master> head -1 master.backup
-	-- SYS_REPLICATION_LOG: COMMIT_ID = 33426, ID = 35074
-
-The SYS_REPLICATION_LOG line provides the replication log metadata needed when starting a new slave. It has two pieces of information:
-
-* **COMMIT_ID**:  This value is the commit sequence number recorded for the most recently executed transaction stored in the transaction log. We can use this value to determine proper commit order within the log. The unique transaction ID cannot be used since that value is assigned when the transaction is started, not when it is committed.
-* **ID**:  This is the unique transaction identifier associated with the most recently executed transaction stored in the transaction log.
-
-Next, steps #3 and #4 must be completed to start the new slave. First, you must start the slave WITHOUT the replication slave plugin enabled, to prevent it from reading from the master until the backup is imported. To start it without the plugin enabled, import your backup, then shutdown the server: ::
-
-	slave> sbin/drizzled --datadir=$PWD/var &
-	slave> drizzle < master.backup
-	slave> drizzle --shutdown
-
-Now that the backup is imported, restart the slave with the replication slave plugin enabled and use a new option, --slave.max-commit-id, to force the slave to begin reading the master's transaction log at the proper location:
-
-	slave> sbin/drizzled --datadir=$PWD/var \
-                                    --plugin-add=slave \
-                                    --slave.config-file=/user/local/etc/slave.cfg \
-                                    --slave.max-commit-id=33426 &
-
-
-We give the --slave.max-commit-id the value from the comment in the master dump file, which defines the maximum COMMIT_ID value (the latest transaction) represented by the slave's contents.
-
-This is the full cycle for a simple replication example. Please see the other Drizzle slave plugin docs for more information on replication and configuration options.

=== modified file 'plugin/slave/module.cc'
--- plugin/slave/module.cc	2012-01-15 20:54:59 +0000
+++ plugin/slave/module.cc	2012-03-18 00:36:19 +0000
@@ -62,7 +62,7 @@
 {
   DRIZZLE_VERSION_ID,
   "slave",
-  "1.0",
+  "1.1",
   "David Shrewsbury",
   N_("Drizzle replication slave"),
   PLUGIN_LICENSE_GPL,

=== modified file 'plugin/zeromq/docs/index.rst'
--- plugin/zeromq/docs/index.rst	2011-10-23 05:45:09 +0000
+++ plugin/zeromq/docs/index.rst	2012-03-18 00:36:19 +0000
@@ -3,51 +3,4 @@
 ZeroMQ
 ======
 
-ZeroMQ is a messaging library that allows you to easily build complex
-communication systems. The ZeroMQ plugin allows drizzle to publish
-transactions to a local PUB socket. Many clients can subscribe to
-these transactions. The first frame of the message sent out is the
-schema name the transaction touched - this enables clients to only
-subscribe to the interesting schemas (note that certain "transactions"
-are without a schema, like SET xyz for example, for these, the first
-frame is empty).
-
-Getting started
----------------
-
-First, install zeromq, get the code from `zeromq.org
-<http://zeromq.org>`_, then you can build drizzle, watch the
-./configure output to verify that drizzle finds the libraries needed.
-
-Now you are good to go, simply start drizzle with --plugin-add=zeromq
-and drizzle will start publishing transactions. The only configuration
-parameter available is:
-
-.. code-block:: bash
-
-  --zeromq.endpoint arg (=tcp://*:9999) - the endpoint to expose.
-
-Now you can write a simple python script to verify that it works,
-something like this will work:
-
-.. code-block:: python
-
-  import zmq
-
-  ctx = zmq.Context()
-  s = ctx.socket(zmq.SUB)
-  s.setsockopt(zmq.SUBSCRIBE, '')
-  s.connect('tcp://localhost:9999')
-  i = 0
-  while True:
-      i = i+1
-      s.recv_multipart()
-      print i
-
-and then you can generate some load:
-
-.. code-block:: bash
-
-  bin/drizzleslap -c 10 -a --auto-generate-sql-add-autoincrement --burnin
-
-which creates 10 threads and generates random queries.
+See :ref:`zeromq_applier`.