Merge lp:~hrvojem/percona-xtradb-cluster/pxc-12 into lp:percona-xtradb-cluster/percona-xtradb-cluster-5.5

[Hrvoje Matijakovic]

[Jay Janssen (jay-janssen)]

"The downside of `mysqldump` and `rsync` is that the cluster becomes *READ-ONLY* while data is being copied from one node to another (SST applies :command:`FLUSH TABLES WITH READ LOCK` command). "

The cluster becomes read-only, or the Donor node? I think only the Donor node (but I could be wrong, I haven't done much with mysqldump and rsync.

"+ $ yum install Percona-XtraDB-Cluster-server Percona-XtraDB-Cluster-client xtrabackup"

xtrabackup is a dependency of PXC server, AFAIK.

"+ wsrep_cluster_address=gcomm://192.168.70.72,192.168.70.73"

Actually, there's no reason why node1's ip can't be in the list, Galera is smart enough to remove it -- and this gives you a setting that you can make identical on all the nodes.

Do we need innodb_locks_unsafe_for_binlog?

I wouldn't necessarily recommend using root for SST, but a separate user. Include a link to the xtrabackup doc about grants necessary.

Why are you not setting wsrep_cluster_name?

"40 +.. note::
441 +
442 + Starting the cluster using the ``/etc/init.d/myslq start --wsrep-cluster-address="gcomm://"`` will only work on RedHat based distributions (like CentOS). This method won't work on Debian/Ubuntu due to difference in the init script.
443 +"

Spelling of 'mysql' ^^
How would you bootstrap if you can't use the init script?

"+This method uses :program:`rsync` to copy files from donor to the joining node. In some cases this can be faster than using the |XtraBackup| but requires the global data lock which will block writes to the cluster. This method doesn't require username/password credentials to be set up in the variable :variable:`wsrep_sst_auth`."

- Again, I think you mean 'node' here, not 'cluster'.

[Jay Janssen (jay-janssen)]

Per previous comments

review: Needs Fixing

[Hrvoje Matijakovic (hrvojem)]

Fixed everything except:

> "40 +.. note::
> 441 +
> 442 + Starting the cluster using the ``/etc/init.d/myslq start --wsrep-
> cluster-address="gcomm://"`` will only work on RedHat based distributions
> (like CentOS). This method won't work on Debian/Ubuntu due to difference in
> the init script.
> 443 +"

> How would you bootstrap if you can't use the init script?
>

This is already described in the 3rd paragraph on that page:
http://www.percona.com/doc/percona-xtradb-cluster/manual/bootstrap.html

Let me know if you still think this should be described for other distros.

[Jay Janssen (jay-janssen)]

On Mar 7, 2013, at 9:16 AM, Hrvoje Matijakovic <email address hidden> wrote:

> This is already described in the 3rd paragraph on that page:
> http://www.percona.com/doc/percona-xtradb-cluster/manual/bootstrap.html

Then I suggest linking to it there --- "you can also bootstrap by modifying the config file as described at this link..."

Jay Janssen, MySQL Consulting Lead, Percona
http://about.me/jay.janssen
Percona Live in Santa Clara, CA April 22nd-25th 2013
http://www.percona.com/live/mysql-conference-2013/

[Raghavendra D Prabhu (raghavendra-prabhu)]

> "The downside of `mysqldump` and `rsync` is that the cluster becomes *READ-
> ONLY* while data is being copied from one node to another (SST applies
> :command:`FLUSH TABLES WITH READ LOCK` command). "

> The cluster becomes read-only, or the Donor node? I think only the Donor node
> (but I could be wrong, I haven't done much with mysqldump and rsync.

Thats right, by default only the donor node becomes read-only.
However, keep in mind this parameter:
gcs.sync_donor Should the rest of the cluster keep in sync with the donor? “Yes” means that if the donor is blocked by state transfer, the whole cluster is blocked with it.

from http://www.codership.com/wiki/doku.php?id=galera_parameters
(may be you can reference here to the variable index we have)

Now, with Xtrabackup too, it will be read-only too but for a
shorter while:

=====================================================================
        # make a prep copy before locking tables, if using rsync
        backup_files(1);

        # flush tables with read lock
        mysql_lockall();
    }

    if ($option_slave_info) {
        write_slave_info();
    }

    # backup non-InnoDB files and tables
    # (or finalize the backup by syncing changes if using rsync)
    backup_files(0);

    # resume ibbackup and wait till log copying is finished
    resume_ibbackup();

    # release read locks on all tables
    mysql_unlockall() if !$option_no_lock;

=====================================================================

So, under FTWRL, Xtrabackup does:

    1) Write binlog info, galera info and slave info

    2) A lighter rsync (which is a good optimization)

    3) Resume the ibbackup for copying the log files from a
    particular checkpoint

Along same lines, rsync SST does:

    a) FTWRL
    b) Disable writes (with google' patch)
    c) Full rsync
    d) Allow writes
    e) Unlock tables

Now, if you compare both. in Xtrabackup 2 + 3 should take less
time than c in rsync SST, hence being under FTWRL for a shorter
while. (It is better if Xtrabackup devs/AlexeyK also review the
above).

So, I think rather than saying read-only only for mysqldump and
rsync SST, we should compare the duration of FTWRL in both and
hence, the duration upto which the node is read-locked.

> "+ $ yum install Percona-XtraDB-Cluster-server Percona-XtraDB-Cluster-client
> xtrabackup"

> xtrabackup is a dependency of PXC server, AFAIK.

> "+ wsrep_cluster_address=gcomm://192.168.70.72,192.168.70.73"

> Actually, there's no reason why node1's ip can't be in the list, Galera is
> smart enough to remove it -- and this gives you a setting that you can make
> identical on all the nodes.

> Do we need innodb_locks_unsafe_for_binlog?

> I wouldn't necessarily recommend using root for SST, but a separate user.
> Include a link to the xtrabackup doc about grants necessary.

> Why are you not setting wsrep_cluster_name?

> "40 +.. note::
> 441 +
> 442 + Starting the cluster using the ``/etc/init.d/myslq start --wsrep-
> cluster-address="gcomm://"`` will only work on RedHat based distributions
> (like CentOS). This method won't work on Debian/Ubuntu due to difference in
> the init script...

review: Needs Fixing

[Raghavendra D Prabhu (raghavendra-prabhu)]

> Thats right, by default only the donor node becomes read-only.
> However, keep in mind this parameter:
> gcs.sync_donor Should the rest of the cluster keep in sync with the donor?
> “Yes” means that if the donor is blocked by state transfer, the whole cluster
> is blocked with it.

> from http://www.codership.com/wiki/doku.php?id=galera_parameters
> (may be you can reference here to the variable index we have)

> Now, with Xtrabackup too, it will be read-only too but for a
> shorter while:

> =====================================================================
> # make a prep copy before locking tables, if using rsync
> backup_files(1);

> # flush tables with read lock
> mysql_lockall();
> }

> if ($option_slave_info) {
> write_slave_info();
> }

> # backup non-InnoDB files and tables
> # (or finalize the backup by syncing changes if using rsync)
> backup_files(0);

> # resume ibbackup and wait till log copying is finished
> resume_ibbackup();

> # release read locks on all tables
> mysql_unlockall() if !$option_no_lock;

> =====================================================================

> So, under FTWRL, Xtrabackup does:

> 1) Write binlog info, galera info and slave info

> 2) A lighter rsync (which is a good optimization)

> 3) Resume the ibbackup for copying the log files from a
> particular checkpoint

> Along same lines, rsync SST does:

> a) FTWRL
> b) Disable writes (with google' patch)
> c) Full rsync
> d) Allow writes
> e) Unlock tables

> Now, if you compare both. in Xtrabackup 2 + 3 should take less
> time than c in rsync SST, hence being under FTWRL for a shorter
> while. (It is better if Xtrabackup devs/AlexeyK also review the
> above).

> So, I think rather than saying read-only only for mysqldump and
> rsync SST, we should compare the duration of FTWRL in both and
> hence, the duration upto which the node is read-locked.

Sorry I misread the original draft, what I mean here is that for
Xtrabackup it may not be read-only only for the duration of syncing of
FRM files but 1,2,3 above - so this part requires expansion.

[Jay Janssen (jay-janssen)]

Assuming all my comments are integrated, I approve (didn't read the whole diff again though)

review: Approve

[Raghavendra D Prabhu (raghavendra-prabhu)]

Looks good. Add the 'default No' part as discussed.

review: Approve

=== modified file 'doc-pxc/source/glossary.rst'
--- doc-pxc/source/glossary.rst	2013-02-01 13:41:34 +0000
+++ doc-pxc/source/glossary.rst	2013-03-12 09:35:25 +0000
@@ -23,7 +23,7 @@
      Incremental State Transfer. Functionality which instead of whole state snapshot can catch up with te group by receiving the missing writesets, but only if the writeset is still in the donor's writeset cache.
 
    SST
-     State Snapshot Transfer is the full copy of data from one node to another. It's used when a new node joins the cluster, it has to transfer data from existing node. There are three methods of SST available in Percona XtraDB Cluster: :program:`mysqldump`, :program:`rsync` and :program:`xtrabackup` (Percona |XtraBackup| with support of XtraDB Cluster will be released soon, currently you need to use our `source code repository <http://www.percona.com/doc/percona-xtrabackup/installation/compiling_xtrabackup.html>`_). The downside of `mysqldump` and `rsync` is that your cluster becomes *READ-ONLY* while data is being copied from one node to another (SST applies :command:`FLUSH TABLES WITH READ LOCK` command). Xtrabackup SST does not require :command:`READ LOCK` for the entire syncing process, only for syncing |.FRM| files (the same as with regular backup).
+     State Snapshot Transfer is the full copy of data from one node to another. It's used when a new node joins the cluster, it has to transfer data from existing node. There are three methods of SST available in Percona XtraDB Cluster: :program:`mysqldump`, :program:`rsync` and :program:`xtrabackup`. The downside of `mysqldump` and `rsync` is that the node becomes *READ-ONLY* while data is being copied from one node to another (SST applies :command:`FLUSH TABLES WITH READ LOCK` command). Xtrabackup SST does not require :command:`READ LOCK` for the entire syncing process, only for syncing the |MySQL| system tables and writing the information about the binlog, galera and slave information (same as the regular XtraBackup backup). State snapshot transfer method can be configured with the :variable:`wsrep_sst_method` variable.
 
    UUID 
       Universally Unique IDentifier which uniquely identifies the state and the sequence of changes node undergoes.

=== added file 'doc-pxc/source/howtos/cenots_howto.rst'
--- doc-pxc/source/howtos/cenots_howto.rst	1970-01-01 00:00:00 +0000
+++ doc-pxc/source/howtos/cenots_howto.rst	2013-03-12 09:35:25 +0000
@@ -0,0 +1,299 @@
+.. _centos_howto:
+
+Installing Percona XtraDB Cluster on *CentOS*
+=============================================
+
+This tutorial will show how to install the |Percona XtraDB Cluster| on three CentOS 6.3 servers, using the packages from Percona repositories.
+
+This cluster will be assembled of three servers/nodes: ::
+ 
+  node #1
+  hostname: percona1
+  IP: 192.168.70.71
+
+  node #2
+  hostname: percona2
+  IP: 192.168.70.72
+
+  node #3
+  hostname: percona3
+  IP: 192.168.70.73
+
+Prerequisites 
+-------------
+
+ * All three nodes have a CentOS 6.3 installation. 
+ 
+ * Firewall has been set up to allow connecting to ports 3306, 4444, 4567 and 4568
+
+ * SELinux is disabled
+
+Installation
+------------
+
+Percona repository should be set up as described in the :ref:`yum-repo` guide. To enable the repository following command should be used: :: 
+
+  $ rpm -Uhv http://www.percona.com/downloads/percona-release/percona-release-0.0-1.x86_64.rpm
+
+Following command will install |Percona XtraDB Cluster| packages: :: 
+
+  $ yum install Percona-XtraDB-Cluster-server Percona-XtraDB-Cluster-client
+
+When these two commands have been executed successfully on all three nodes |Percona XtraDB Cluster| is installed.
+
+Configuring the nodes
+---------------------
+
+Individual nodes should be configured to be able to bootstrap the cluster. More details about bootstrapping the cluster can be found in the :ref:`bootstrap` guide.
+
+Configuration file :file:`/etc/my.cnf` for the first node should look like: ::
+
+  [mysqld]
+
+  datadir=/var/lib/mysql
+  user=mysql
+
+  # Path to Galera library
+  wsrep_provider=/usr/lib64/libgalera_smm.so
+
+  # Cluster connection URL contains the IPs of node#1, node#2 and node#3
+  wsrep_cluster_address=gcomm://192.168.70.71,192.168.70.72,192.168.70.73
+
+  # In order for Galera to work correctly binlog format should be ROW
+  binlog_format=ROW
+
+  # MyISAM storage engine has only experimental support
+  default_storage_engine=InnoDB
+
+  # This is a recommended tuning variable for performance
+  innodb_locks_unsafe_for_binlog=1
+
+  # This changes how InnoDB autoincrement locks are managed and is a requirement for Galera
+  innodb_autoinc_lock_mode=2
+
+  # Node #1 address
+  wsrep_node_address=192.168.70.71
+
+  # SST method
+  wsrep_sst_method=xtrabackup
+
+  # Cluster name
+  wsrep_cluster_name=my_centos_cluster
+
+  # Authentication for SST method
+  wsrep_sst_auth="sstuser:s3cret"
+
+
+After this, first node can be started with the following command: ::
+
+  [root@percona1 ~]# /etc/init.d/mysql start --wsrep-cluster-address="gcomm://"
+ 
+This command will start the cluster with initial :variable:`wsrep_cluster_address` set to ``gcomm://``. This way the cluster will be bootstrapped and in case the node or |MySQL| have to be restarted later, there would be no need to change the configuration file.
+
+After the first node has been started, cluster status can be checked by: 
+
+.. code-block:: mysql 
+
+  mysql> show status like 'wsrep%';
+  +----------------------------+--------------------------------------+
+  | Variable_name              | Value                                |
+  +----------------------------+--------------------------------------+
+  | wsrep_local_state_uuid     | c2883338-834d-11e2-0800-03c9c68e41ec |
+  ...
+  | wsrep_local_state          | 4                                    |
+  | wsrep_local_state_comment  | Synced                               |
+  ...
+  | wsrep_cluster_size         | 1                                    |
+  | wsrep_cluster_status       | Primary                              |
+  | wsrep_connected            | ON                                   |
+  ...
+  | wsrep_ready                | ON                                   |
+  +----------------------------+--------------------------------------+
+  40 rows in set (0.01 sec)
+
+This output shows that the cluster has been successfully bootstrapped. 
+
+It's recommended not to leave the empty password for the root account. Password can be changed with: 
+
+.. code-block:: mysql 
+
+  mysql@percona1> UPDATE mysql.user SET password=PASSWORD("Passw0rd") where user='root';
+  mysql@percona1> FLUSH PRIVILEGES;
+
+In order to perform successful :ref:`state_snapshot_transfer` using |XtraBackup| new user needs to be set up with proper `privileges <http://www.percona.com/doc/percona-xtrabackup/innobackupex/privileges.html#permissions-and-privileges-needed>`_: 
+
+.. code-block:: mysql
+
+  mysql@percona1> CREATE USER 'sstuser'@'localhost' IDENTIFIED BY 's3cret';
+  mysql@percona1> GRANT RELOAD, LOCK TABLES, REPLICATION CLIENT ON *.* TO 'sstuser'@'localhost';
+  mysql@percona1> FLUSH PRIVILEGES;
+
+
+.. note:: 
+
+ MySQL root account can also be used for setting up the SST with Percona XtraBackup, but it's recommended to use a different (non-root) user for this.
+
+Configuration file :file:`/etc/my.cnf` on the second node (``percona2``) should look like this: ::
+
+  [mysqld]
+
+  datadir=/var/lib/mysql
+  user=mysql
+
+  # Path to Galera library
+  wsrep_provider=/usr/lib64/libgalera_smm.so
+
+  # Cluster connection URL contains IPs of node#1, node#2 and node#3
+  wsrep_cluster_address=gcomm://192.168.70.71,192.168.70.72,192.168.70.73
+
+  # In order for Galera to work correctly binlog format should be ROW
+  binlog_format=ROW
+
+  # MyISAM storage engine has only experimental support
+  default_storage_engine=InnoDB
+
+  # This is a recommended tuning variable for performance
+  innodb_locks_unsafe_for_binlog=1
+
+  # This changes how InnoDB autoincrement locks are managed and is a requirement for Galera
+  innodb_autoinc_lock_mode=2
+
+  # Node #2 address
+  wsrep_node_address=192.168.70.72
+
+  # SST method
+  wsrep_sst_method=xtrabackup
+
+  #Authentication for SST method
+  wsrep_sst_auth="sstuser:s3cret"
+ 
+Second node can be started with the following command: ::
+
+  [root@percona2 ~]# /etc/init.d/mysql start
+
+After the server has been started it should receive the state snapshot transfer automatically. This means that the second node won't have the empty root password anymore. In order to connect to the cluster and check the status changed root password from the first node should be used. Cluster status can now be checked on both nodes. This is the example from the second node (``percona2``): 
+
+.. code-block:: mysql 
+
+  mysql> show status like 'wsrep%';
+  +----------------------------+--------------------------------------+
+  | Variable_name              | Value                                |
+  +----------------------------+--------------------------------------+
+  | wsrep_local_state_uuid     | c2883338-834d-11e2-0800-03c9c68e41ec |
+  ...
+  | wsrep_local_state          | 4                                    |
+  | wsrep_local_state_comment  | Synced                               |
+  ...
+  | wsrep_cluster_size         | 2                                    |
+  | wsrep_cluster_status       | Primary                              |
+  | wsrep_connected            | ON                                   |
+  ...
+  | wsrep_ready                | ON                                   |
+  +----------------------------+--------------------------------------+
+  40 rows in set (0.01 sec)
+
+This output shows that the new node has been successfully added to the cluster. 
+
+MySQL configuration file :file:`/etc/my.cnf` on the third node (``percona3``) should look like this: ::
+
+  [mysqld]
+
+  datadir=/var/lib/mysql
+  user=mysql
+
+  # Path to Galera library
+  wsrep_provider=/usr/lib64/libgalera_smm.so
+
+  # Cluster connection URL contains IPs of node#1, node#2 and node#3
+  wsrep_cluster_address=gcomm://192.168.70.71,192.168.70.72,192.168.70.73
+
+  # In order for Galera to work correctly binlog format should be ROW
+  binlog_format=ROW
+
+  # MyISAM storage engine has only experimental support
+  default_storage_engine=InnoDB
+
+  # This is a recommended tuning variable for performance
+  innodb_locks_unsafe_for_binlog=1
+
+  # This changes how InnoDB autoincrement locks are managed and is a requirement for Galera
+  innodb_autoinc_lock_mode=2
+
+  # Node #3 address
+  wsrep_node_address=192.168.70.73
+
+  # SST method
+  wsrep_sst_method=xtrabackup
+
+  #Authentication for SST method
+  wsrep_sst_auth="sstuser:s3cret"
+
+Third node can now be started with the following command: :: 
+
+  [root@percona3 ~]# /etc/init.d/mysql start
+
+After the server has been started it should receive the SST same as the second node. Cluster status can now be checked on both nodes. This is the example from the third node (``percona3``): 
+
+.. code-block:: mysql 
+
+  mysql> show status like 'wsrep%';
+  +----------------------------+--------------------------------------+
+  | Variable_name              | Value                                |
+  +----------------------------+--------------------------------------+
+  | wsrep_local_state_uuid     | c2883338-834d-11e2-0800-03c9c68e41ec |
+  ...
+  | wsrep_local_state          | 4                                    |
+  | wsrep_local_state_comment  | Synced                               |
+  ...
+  | wsrep_cluster_size         | 3                                    |
+  | wsrep_cluster_status       | Primary                              |
+  | wsrep_connected            | ON                                   |
+  ...
+  | wsrep_ready                | ON                                   |
+  +----------------------------+--------------------------------------+
+  40 rows in set (0.01 sec)
+
+This output confirms that the third node has joined the cluster.
+
+Testing the replication
+-----------------------
+
+Although the password change from the first node has replicated successfully, this example will show that writing on any node will replicate to the whole cluster. In order to check this, new database will be created on second node and table for that database will be created on the third node.
+
+Creating the new database on the second node: 
+
+.. code-block:: mysql 
+
+  mysql@percona2> CREATE DATABASE percona;
+  Query OK, 1 row affected (0.01 sec)
+
+Creating the ``example`` table on the third node: 
+  
+.. code-block:: mysql 
+
+  mysql@percona3> USE percona;
+  Database changed
+
+  mysql@percona3> CREATE TABLE example (node_id INT PRIMARY KEY, node_name VARCHAR(30));
+  Query OK, 0 rows affected (0.05 sec)
+
+Inserting records on the first node: 
+
+.. code-block:: mysql 
+
+  mysql@percona1> INSERT INTO percona.example VALUES (1, 'percona1');
+  Query OK, 1 row affected (0.02 sec)
+
+Retrieving all the rows from that table on the second node: 
+
+.. code-block:: mysql 
+
+  mysql@percona2> SELECT * FROM percona.example;
+  +---------+-----------+
+  | node_id | node_name |
+  +---------+-----------+
+  |       1 | percona1  |
+  +---------+-----------+
+  1 row in set (0.00 sec)
+
+This small example shows that all nodes in the cluster are synchronized and working as intended.

=== modified file 'doc-pxc/source/index.rst'
--- doc-pxc/source/index.rst	2013-02-01 13:41:34 +0000
+++ doc-pxc/source/index.rst	2013-03-12 09:35:25 +0000
@@ -36,7 +36,6 @@
    :glob:
 
    intro
-   resources
    limitation
 
 Installation
@@ -67,6 +66,7 @@
    :glob:
 
    manual/bootstrap
+   manual/state_snapshot_transfer
    manual/restarting_nodes
    manual/failover
    manual/monitoring
@@ -78,6 +78,7 @@
    :maxdepth: 1
    :glob:
   
+   howtos/cenots_howto
    howtos/singlebox
    howtos/3nodesec2
    howtos/haproxy
@@ -95,6 +96,7 @@
    release-notes/release-notes_index
    wsrep-status-index
    wsrep-system-index
+   wsrep-files-index
    faq
    glossary
 

=== modified file 'doc-pxc/source/installation.rst'
--- doc-pxc/source/installation.rst	2013-02-01 13:41:34 +0000
+++ doc-pxc/source/installation.rst	2013-03-12 09:35:25 +0000
@@ -24,28 +24,52 @@
    installation/yum_repo
    installation/apt_repo
 
-|Percona| provides repositories for :program:`yum` (``RPM`` packages for *Red Hat*, *CentOS*, *Amazon Linux AMI*, and *Fedora*) and :program:`apt` (:file:`.deb` packages for *Ubuntu* and *Debian*) for software such as |Percona Server|, |XtraDB|, |XtraBackup|, and *Percona Toolkit*. This makes it easy to install and update your software and its dependencies through your operating system's package manager.
+|Percona| provides repositories for :program:`yum` (``RPM`` packages for *Red Hat*, *CentOS* and *Amazon Linux AMI*) and :program:`apt` (:file:`.deb` packages for *Ubuntu* and *Debian*) for software such as |Percona Server|, |XtraDB|, |XtraBackup|, and *Percona Toolkit*. This makes it easy to install and update your software and its dependencies through your operating system's package manager.
 
 This is the recommend way of installing where possible.
 
+``YUM``-Based Systems
+---------------------
+
+Once the repository is set up, use the following commands: ::
+
+  $ yum install Percona-XtraDB-Cluster-server Percona-XtraDB-Cluster-client
+
+More detailed example of the |Percona XtraDB Cluster| installation and configuration can be seen in :ref:`centos_howto` tutorial.
+
+``DEB``-Based Systems
+---------------------
+
+Once the repository is set up, use the following commands: ::
+
+  $ sudo apt-get install percona-xtradb-cluster-server-5.5 percona-xtradb-cluster-client-5.5
+
+Prerequisites
+=============
+
+In order for |Percona XtraDB Cluster| to work correctly firewall has to be set up to allow connections on the following ports: 3306, 4444, 4567 and 4568. |Percona XtraDB Cluster| currently doesn't work with ``SELinux`` or ``apparmor`` so they should be disabled, otherwise individual nodes won't be able to communicate and form the cluster.
 
 Initial configuration
 =====================
 
-In order to start using XtraDB Cluster, you need to configure my.cnf file. Following options are needed: ::
+In order to start using the |Percona XtraDB Cluster|, following options are needed in the |MySQL| configuration file :file:`my.cnf`: ::
 
   [mysqld]
+  
   wsrep_provider — a path to Galera library.
-  wsrep_cluster_address — cluster connection URL.
-  binlog_format=ROW
-  default_storage_engine=InnoDB
-  innodb_autoinc_lock_mode=2
-  innodb_locks_unsafe_for_binlog=1
-
-Additional parameters to tune: ::
-
-  wsrep_slave_threads   # specifies amount of threads to apply events
-  wsrep_sst_method
+  wsrep_cluster_address — Cluster connection URL containing the IPs of other nodes in the cluster
+  wsrep_sst_method - method used for the state snapshot transfer 
+  
+  binlog_format=ROW - In order for Galera to work correctly binlog format should be ROW
+  default_storage_engine=InnoDB - MyISAM storage engine has only experimental support
+  innodb_locks_unsafe_for_binlog=1 - This is a recommended tuning variable for performance
+  innodb_autoinc_lock_mode=2 - This changes how InnoDB autoincrement locks are managed
+
+Additional parameters to specify: ::
+
+  wsrep_sst_auth=user:password   
+
+If any other :ref:`state_snapshot_transfer` method beside the :program:`rsync` is specified in the :variable:`wsrep_sst_method`, credentials for |SST| need to be specified.
 
 Example: ::
 
@@ -53,23 +77,10 @@
   wsrep_cluster_address=gcomm://10.11.12.206
   wsrep_slave_threads=8
   wsrep_sst_method=rsync
-  #wsrep_sst_method=xtrabackup - alternative way to do SST
-  wsrep_cluster_name=percona_test_cluster
   binlog_format=ROW
   default_storage_engine=InnoDB
+  innodb_locks_unsafe_for_binlog=1
   innodb_autoinc_lock_mode=2
-  innodb_locks_unsafe_for_binlog=1
 
 Detailed list of variables can be found in :ref:`wsrep_system_index` and :ref:`wsrep_status_index`.
 
-Install XtraBackup SST method
-==============================
-
-To use Percona XtraBackup for State Transfer method (copy snapshot of data between nodes)
-you can use the regular xtrabackup package with the script what supports Galera information.
-You can take *innobackupex* script from source code `innobackupex <http://bazaar.launchpad.net/~percona-dev/percona-xtrabackup/galera-info/view/head:/innobackupex>`_.
-
-To inform node to use xtrabackup you need to specify in my.cnf: ::
-
-  wsrep_sst_method=xtrabackup 
-

=== modified file 'doc-pxc/source/installation/yum_repo.rst'
--- doc-pxc/source/installation/yum_repo.rst	2013-02-01 13:41:34 +0000
+++ doc-pxc/source/installation/yum_repo.rst	2013-03-12 09:35:25 +0000
@@ -6,7 +6,7 @@
 
 The |Percona| :program:`yum` repository supports popular *RPM*-based operating systems, including the *Amazon Linux AMI*.
 
-The easiest way to install the *Percona Yum* repository is to install an *RPM* that configures :program:`yum` and installs the `Percona GPG key <http://www.percona.com/downloads/RPM-GPG-KEY-percona>`_. You can also do the installation manually.
+The easiest way to install the *Percona Yum* repository is to install an *RPM* that configures :program:`yum` and installs the `Percona GPG key <https://www.percona.com/downloads/RPM-GPG-KEY-percona>`_. Installation can also be done manually.
 
 Automatic Installation
 =======================

=== modified file 'doc-pxc/source/manual/bootstrap.rst'
--- doc-pxc/source/manual/bootstrap.rst	2013-02-01 13:41:34 +0000
+++ doc-pxc/source/manual/bootstrap.rst	2013-03-12 09:35:25 +0000
@@ -30,10 +30,14 @@
 
 In case cluster that's being bootstrapped has already been set up before, and to avoid editing the :file:`my.cnf` twice to change the :variable:`wsrep_cluster_address` to ``gcomm://`` and then to change it back to other node addresses, first node can be started with: :: 
  
-  /etc/init.d/myslq start --wsrep-cluster-address="gcomm://"
+  /etc/init.d/mysql start --wsrep-cluster-address="gcomm://"
 
 This way values in :file:`my.cnf` would remain unchanged. Next time node is restarted it won't require updating the configuration file. This can be useful in case cluster has been previously set up and for some reason all nodes went down and the cluster needs to be bootstrapped again. 
 
+.. note:: 
+
+ Starting the cluster using the ``/etc/init.d/mysql start --wsrep-cluster-address="gcomm://"`` will only work on RedHat based distributions (like CentOS). This method won't work on Debian/Ubuntu due to difference in the init script.
+
 Other Reading
 =============
 

=== added file 'doc-pxc/source/manual/state_snapshot_transfer.rst'
--- doc-pxc/source/manual/state_snapshot_transfer.rst	1970-01-01 00:00:00 +0000
+++ doc-pxc/source/manual/state_snapshot_transfer.rst	2013-03-12 09:35:25 +0000
@@ -0,0 +1,42 @@
+.. _state_snapshot_transfer:
+
+=========================
+ State Snapshot Transfer
+=========================
+
+State Snapshot Transfer is a full data copy from one node (donor) to the joining node (joiner). It's used when a new node joins the cluster. In order to be synchronized with the cluster, new node has to transfer data from the node that is already part of the cluster.  
+There are three methods of SST available in Percona XtraDB Cluster: :program:`mysqldump`, :program:`rsync` and :program:`xtrabackup`. The downside of `mysqldump` and `rsync` is that the donor node becomes *READ-ONLY* while data is being copied from one node to another (SST applies :command:`FLUSH TABLES WITH READ LOCK` command). Xtrabackup SST does not require :command:`READ LOCK` for the entire syncing process, only for syncing the |MySQL| system tables and writing the information about the binlog, galera and slave information (same as the regular XtraBackup backup). State snapshot transfer method can be configured with the :variable:`wsrep_sst_method` variable.
+
+.. note:: 
+
+ If the variable :variable:`gcs.sync_donor` is set to ``Yes`` (default ``No``), whole cluster will get blocked if the donor is blocked by the State Snapshot Transfer and not just the donor node.
+
+Using *Percona Xtrabackup*
+==========================
+
+This is the least blocking method as it locks the tables only to copy the |MyISAM| system tables. |XtraBackup| is run locally on the donor node, so it's important that the correct user credentials are set up on the donor node. In order for PXC to perform the SST using the |XtraBackup|, credentials for connecting to the donor node need to be set up in the variable :variable:`wsrep_sst_auth`. Beside the credentials, one more important thing is that the :term:`datadir` needs to be specified in the server configuration file :file:`my.cnf`, otherwise the transfer process will fail.
+
+More information about the required credentials can be found in the |XtraBackup| `manual <http://www.percona.com/doc/percona-xtrabackup/innobackupex/privileges.html#permissions-and-privileges-needed>`_. Easy way to test if the credentials will work is to run the |innobackupex| on the donor node with the username and password specified in the variable :variable:`wsrep_sst_auth`. For example, if the value of the :variable:`wsrep_sst_auth` is ``root:Passw0rd`` |innobackupex| command should look like: :: 
+
+  innobackupex --user=root --password=Passw0rd /tmp/
+ 
+Script used for this method can be found in :file:`/usr/bin/wsrep_sst_xtrabackup` and it's provided with the |PXC| binary packages.
+
+Using ``mysqldump``
+===================
+
+This method uses the standard :program:`mysqldump` to dump all the databases from the donor node and import them to the joining node. For this method to work :variable:`wsrep_sst_auth` needs to be set up with the root credentials. This method is the slowest one and it also performs the global lock while doing the |SST| which will block writes to the donor node.
+
+Script used for this method can be found in :file:`/usr/bin/wsrep_sst_mysqldump` and it's provided with the |PXC| binary packages.
+
+Using ``rsync``
+===============
+
+This method uses :program:`rsync` to copy files from donor to the joining node. In some cases this can be faster than using the |XtraBackup| but requires the global data lock which will block writes to the donor node. This method doesn't require username/password credentials to be set up in the variable :variable:`wsrep_sst_auth`.
+
+Script used for this method can be found in :file:`/usr/bin/wsrep_sst_rsync` and it's provided with the |PXC| binary packages.
+
+Other Reading
+=============
+
+* `SST Methods for MySQL <http://www.codership.com/wiki/doku.php?id=sst_mysql>`_

=== removed file 'doc-pxc/source/resources.rst'
--- doc-pxc/source/resources.rst	2013-01-25 07:37:41 +0000
+++ doc-pxc/source/resources.rst	1970-01-01 00:00:00 +0000
@@ -1,29 +0,0 @@
-.. _resources:
-
-=========
-Resources
-=========
-
-In general there are 4 resources that need to be different when you want to run several MySQL/Galera nodes on one host:
-
-1) data directory
-2) mysql client port and/or address
-3) galera replication listen port and/or address
-4) receive address for state snapshot transfer
-
-and later incremental state transfer receive address will be added to the bunch. (I know, it is kinda a lot, but we don't see how it can be meaningfully reduced yet).
-
-The first two are the usual mysql stuff.
-
-You figured out the third. It is also possible to pass it via: ::
-
-   wsrep_provider_options="gmcast.listen_addr=tcp://127.0.0.1:5678"
-
-as most other Galera options. This may save you some extra typing.
-
-The fourth one is :option:`wsrep_sst_receive_address`. This is the address at which the node will be listening for and receiving the state. Note that in galera cluster joining nodes are waiting for connections from donors. It goes contrary to tradition and seems to confuse people time and again, but there are good reasons it was made like that.
-
-If you use mysqldump SST it should be the same as this mysql client connection address plus you need to set :option:`wsrep_sst_auth` variable to hold user:password pair. The user should be privileged enough to read system tables from donor and create system tables on this node. For simplicity that could be just the root user. Note that it also means that you need to properly set up the privileges on the new node before attempting to join the cluster. If you use |xtrabackup| as SST method, it will use /usr/bin/wsrep_sst_xtrabackup provided in Percona-XtraDB-Cluster-server package. And this script also needs user password if you have a password for root@localhost.
-
-If you use rsync SST, :option:`wsrep_sst_auth` is not necessary unless your SST script makes use of it. :option:`wsrep_sst_receive_address` can be anything local (it may even be the same on all nodes provided you'll be starting them one at a time).
-

=== added file 'doc-pxc/source/wsrep-files-index.rst'
--- doc-pxc/source/wsrep-files-index.rst	1970-01-01 00:00:00 +0000
+++ doc-pxc/source/wsrep-files-index.rst	2013-03-12 09:35:25 +0000
@@ -0,0 +1,68 @@
+.. _wsrep_file_index:
+
+===============================
+ Index of files created by PXC
+===============================
+
+* :file:`GRA_*.log`
+   These files contain binlog events in ROW format representing the failed transaction. That means that the slave thread was not able to apply one of the transactions. For each of those file, a corresponding warning or error message is present in the mysql error log file. Those error can also be false positives like a bad ``DDL`` statement (dropping  a table that doesn't exists for example) and therefore nothing to worry about. However it's always recommended to check these log to understand what's is happening.
+
+   To be able to analyze these files `binlog header <http://www.mysqlperformanceblog.com/wp-content/uploads/2012/12/GRA-header.zip>`_ needs to be added to the log file. :: 
+  
+      $ cat GRA-header > /var/lib/mysql/GRA_1_2-bin.log
+      $ cat /var/lib/mysql/GRA_1_2.log >> /var/lib/mysql/GRA_1_2-bin.log
+      $ mysqlbinlog -vvv /var/lib/mysql/GRA_1_2-bin.log 
+      /*!40019 SET @@session.max_insert_delayed_threads=0*/;
+      /*!50003 SET @OLD_COMPLETION_TYPE=@@COMPLETION_TYPE,COMPLETION_TYPE=0*/;
+      DELIMITER /*!*/;
+      # at 4
+      #120715  9:45:56 server id 1  end_log_pos 107 	Start: binlog v 4, server v 5.5.25-debug-log created 120715  9:45:56 at startup
+      # Warning: this binlog is either in use or was not closed properly.
+      ROLLBACK/*!*/;
+      BINLOG '
+      NHUCUA8BAAAAZwAAAGsAAAABAAQANS41LjI1LWRlYnVnLWxvZwAAAAAAAAAAAAAAAAAAAAAAAAAA
+      AAAAAAAAAAAAAAAAAAA0dQJQEzgNAAgAEgAEBAQEEgAAVAAEGggAAAAICAgCAA==
+      '/*!*/;
+      # at 107
+      #130226 11:48:50 server id 1  end_log_pos 83 	Query	thread_id=3	exec_time=0	error_code=0
+      use `test`/*!*/;
+      SET TIMESTAMP=1361875730/*!*/;
+      SET @@session.pseudo_thread_id=3/*!*/;
+      SET @@session.foreign_key_checks=1, @@session.sql_auto_is_null=0, @@session.unique_checks=1, @@session.autocommit=1/*!*/;
+      SET @@session.sql_mode=1437073440/*!*/;
+      SET @@session.auto_increment_increment=3, @@session.auto_increment_offset=2/*!*/;
+      /*!\C utf8 *//*!*/;
+      SET @@session.character_set_client=33,@@session.collation_connection=33,@@session.collation_server=8/*!*/;
+      SET @@session.lc_time_names=0/*!*/;
+      SET @@session.collation_database=DEFAULT/*!*/;
+      drop table test
+      /*!*/;
+      DELIMITER ;
+      # End of log file
+      ROLLBACK /* added by mysqlbinlog */;
+      /*!50003 SET COMPLETION_TYPE=@OLD_COMPLETION_TYPE*/;
+
+   This information can be used for checking the |MySQL| error log for the corresponding error message. :: 
+
+     130226 11:48:50 [ERROR] Slave SQL: Error 'Unknown table 'test'' on query. Default database: 'test'. Query: 'drop table test', Error_code: 1051
+     130226 11:48:50 [Warning] WSREP: RBR event 1 Query apply warning: 1, 3
+     130226 11:48:50 [Warning] WSREP: Ignoring error for TO isolated action: source: dd40ad88-7ff9-11e2-0800-e93cbffe93d7 version: 2 local: 0 state: APPLYING flags: 65 conn_id: 3 trx_id: -1 seqnos (l: 5, g: 3, s: 2, d: 2, ts: 1361875730070283555)
+  
+   In this example ``DROP TABLE`` statement was executed on a table that doesn't exist.
+
+.. _galera.cache: galera_cache
+
+* :file:`galera.cache`
+   This file is used as a main writeset store. It's implemented as a permanent ring-buffer file that is preallocated on disk when the node is initialized. File size can be controlled with the variable :variable:`gcache.size`. If this value is bigger, more writesets are cached and chances are better that the re-joining node will get |IST| instead of |SST|. Filename can be changed with the :variable:`gcache.name` variable.
+  
+* :file:`grastate.dat`
+   This file contains the Galera state information.
+
+   Example of this file looks like this: ::
+  
+    # GALERA saved state
+    version: 2.1
+    uuid:    1917033b-7081-11e2-0800-707f5d3b106b
+    seqno:   -1
+    cert_index:
+

=== modified file 'doc-pxc/source/wsrep-system-index.rst'
--- doc-pxc/source/wsrep-system-index.rst	2013-02-01 13:41:34 +0000
+++ doc-pxc/source/wsrep-system-index.rst	2013-03-12 09:35:25 +0000
@@ -290,8 +290,9 @@
    :conf: Yes
    :scope: Global
    :dyn: Yes
+   :format: <username>:<password>
 
-This variable should contain the authentication information needed for State Snapshot Transfer. Required information depends on the method selected in the :variable:`wsrep_sst_method`. More information about required authentication can be found in :ref:`resources`. This variable will appear masked in the logs and in the ``SHOW VARIABLES`` query.
+This variable should contain the authentication information needed for State Snapshot Transfer. Required information depends on the method selected in the :variable:`wsrep_sst_method`. More information about required authentication can be found in the :ref:`state_snapshot_transfer` documentation. This variable will appear masked in the logs and in the ``SHOW VARIABLES`` query.
 
 .. variable:: wsrep_sst_donor