Merge lp:~tplavcic/percona-server/bld-212-5.6-tokudb into lp:percona-server/5.6

[Tomislav Plavcic]

This change is for BLD-212 (simplify TokuDB installation).
It adds ps_tokudb_admin script which can install/uninstall TokuDB plugin and disable transparent huge pages.
Mysqld_safe was changed to read thp-setting option from my.cnf

Extensive testing was done in BLD-212 so here I only show how install/uninstall works on centos7.
Test packages are available here:
http://jenkins.percona.com/job/percona-server-5.6-RELEASE/96/

INSTALL
-------
[vagrant@t-centos7-64 ~]$ sudo ps_tokudb_admin -e -u root
Checking transparent huge pages status on the system...
>> Transparent huge pages are enabled (should be disabled).

Checking if thp-setting=never option is already set in config file...
>> Option thp-setting=never is not set in the config file.
>> (needed only if THP is not disabled permanently on the system)

Checking TokuDB plugin status...
>> TokuDB plugin is not installed.

Disabling transparent huge pages for the current session...
>> Successfuly disabled transparent huge pages for this session.

Adding thp-setting=never option into /etc/my.cnf
>> Successfuly added thp-setting=never option into /etc/my.cnf

Installing TokuDB engine...
>> Successfuly installed TokuDB plugin.

mysql> show plugins;
...
| TokuDB | ACTIVE | STORAGE ENGINE | ha_tokudb.so | GPL |
| TokuDB_file_map | ACTIVE | INFORMATION SCHEMA | ha_tokudb.so | GPL |
| TokuDB_fractal_tree_info | ACTIVE | INFORMATION SCHEMA | ha_tokudb.so | GPL |
| TokuDB_fractal_tree_block_map | ACTIVE | INFORMATION SCHEMA | ha_tokudb.so | GPL |
| TokuDB_trx | ACTIVE | INFORMATION SCHEMA | ha_tokudb.so | GPL |
| TokuDB_locks | ACTIVE | INFORMATION SCHEMA | ha_tokudb.so | GPL |
| TokuDB_lock_waits | ACTIVE | INFORMATION SCHEMA | ha_tokudb.so | GPL |
+-------------------------------+----------+--------------------+--------------+---------+

UNINSTALL
---------
[vagrant@t-centos7-64 ~]$ sudo ps_tokudb_admin -d -u root
Checking transparent huge pages status on the system...
>> Transparent huge pages are currently disabled on the system.

Checking if thp-setting=never option is already set in config file...
>> Option thp-setting=never is set in the config file.

Checking TokuDB plugin status...
>> TokuDB plugin is installed.

Removing thp-setting=never option from /etc/my.cnf
>> Successfuly removed thp-setting=never option from /etc/my.cnf

Uninstalling TokuDB plugin...
>> Successfuly uninstalled TokuDB plugin.

[Laurynas Biveinis (laurynas-biveinis)]

It needs a blueprint?

review: Needs Information

[Tomislav Plavcic (tplavcic)]

Created some blueprint - please check.

[Roel Van de Paar (roel11)]

The script does not allow specification of host/socket/port. Thus, it assumes that PS is installed in a standard way (yum/apt), on a standard port, etc.

If the customer has both MS and PS on one box, on different ports, the wrong one may be auto-selected depending on which one is running on the default port, etc.

review: Needs Information

[Roel Van de Paar (roel11)]

Also, I am not sure about removing the INSTALL PLUGIN information from postinst. It may still make sense for many DBA's to quickly hop in a CLI and execute this, rather then using a new tool.

=== modified file 'build-ps/debian/percona-server-tokudb-5.6.files'
--- build-ps/debian/percona-server-tokudb-5.6.files	2014-09-27 13:49:22 +0000
+++ build-ps/debian/percona-server-tokudb-5.6.files	2015-01-15 08:09:15 +0000
@@ -1,3 +1,4 @@
 usr/lib/mysql/plugin/ha_tokudb.so
 usr/lib/mysql/plugin/debug/ha_tokudb.so
 usr/bin/tokuftdump
+usr/bin/ps_tokudb_admin

=== modified file 'build-ps/debian/percona-server-tokudb-5.6.postinst'
--- build-ps/debian/percona-server-tokudb-5.6.postinst	2014-05-20 21:04:08 +0000
+++ build-ps/debian/percona-server-tokudb-5.6.postinst	2015-01-15 08:09:15 +0000
@@ -3,15 +3,9 @@
 # Some postinstall info about TokuDB
 if [ -z "$2" ]; then
 	echo -e "\n\n * This release of Percona Server is distributed with TokuDB storage engine."
-	echo -e " * Run the following commands to enable the TokuDB storage engine in Percona Server:\n"
-	echo -e "\tmysql -e \"INSTALL PLUGIN tokudb SONAME 'ha_tokudb.so';\""
-	echo -e "\tmysql -e \"INSTALL PLUGIN tokudb_file_map SONAME 'ha_tokudb.so';\""
-	echo -e "\tmysql -e \"INSTALL PLUGIN tokudb_fractal_tree_info SONAME 'ha_tokudb.so';\""
-	echo -e "\tmysql -e \"INSTALL PLUGIN tokudb_fractal_tree_block_map SONAME 'ha_tokudb.so';\""
-	echo -e "\tmysql -e \"INSTALL PLUGIN tokudb_trx SONAME 'ha_tokudb.so';\""
-	echo -e "\tmysql -e \"INSTALL PLUGIN tokudb_locks SONAME 'ha_tokudb.so';\""
-	echo -e "\tmysql -e \"INSTALL PLUGIN tokudb_lock_waits SONAME 'ha_tokudb.so';\""
-	echo -e "\n * See http://www.percona.com/doc/percona-server/5.6/tokudb/tokudb_intro.html for more details\n\n"
+	echo -e " * Run the following script to enable the TokuDB storage engine in Percona Server:\n"
+	echo -e "\tps_tokudb_admin --enable -u <mysql_admin_user> -p <mysql_admin_pass>\n"
+	echo -e " * See http://www.percona.com/doc/percona-server/5.6/tokudb/tokudb_intro.html for more details\n\n"
 fi
 
 #DEBHELPER#

=== modified file 'build-ps/percona-server.spec'
--- build-ps/percona-server.spec	2015-01-13 14:38:41 +0000
+++ build-ps/percona-server.spec	2015-01-15 08:09:15 +0000
@@ -651,6 +651,10 @@
 %if 0%{?systemd}
 rm -rf $RBR%{_sysconfdir}/init.d/mysql
 %endif
+# Not needed if TokuDB package is not created
+%if ! %{with tokudb}
+rm -rf $RBR%{_bindir}/ps_tokudb_admin
+%endif
 
 ##############################################################################
 #  Post processing actions, i.e. when installed
@@ -1094,20 +1098,10 @@
 %post -n Percona-Server-tokudb%{product_suffix}
 
 if [ $1 -eq 1 ] ; then
-	echo ""
-	echo "* This release of Percona Server is distributed with TokuDB storage engine."
-	echo "* Run the following commands to enable the TokuDB storage engine in Percona Server:"
-	echo ""
-	echo "mysql -e \"INSTALL PLUGIN tokudb SONAME 'ha_tokudb.so';\""
-	echo "mysql -e \"INSTALL PLUGIN tokudb_file_map SONAME 'ha_tokudb.so';\""
-	echo "mysql -e \"INSTALL PLUGIN tokudb_fractal_tree_info SONAME 'ha_tokudb.so';\""
-	echo "mysql -e \"INSTALL PLUGIN tokudb_fractal_tree_block_map SONAME 'ha_tokudb.so';\""
-	echo "mysql -e \"INSTALL PLUGIN tokudb_trx SONAME 'ha_tokudb.so';\""
-	echo "mysql -e \"INSTALL PLUGIN tokudb_locks SONAME 'ha_tokudb.so';\""
-	echo "mysql -e \"INSTALL PLUGIN tokudb_lock_waits SONAME 'ha_tokudb.so';\""
-	echo ""
-	echo "* See http://www.percona.com/doc/percona-server/5.6/tokudb/tokudb_intro.html for more details"
-	echo ""
+  echo -e "\n\n * This release of Percona Server is distributed with TokuDB storage engine."
+  echo -e " * Run the following script to enable the TokuDB storage engine in Percona Server:\n"
+  echo -e "\tps_tokudb_admin --enable -u <mysql_admin_user> -p <mysql_admin_pass>\n"
+  echo -e " * See http://www.percona.com/doc/percona-server/5.6/tokudb/tokudb_intro.html for more details\n\n"
 fi
 # ----------------------------------------------------------------------------
 %endif
@@ -1375,6 +1369,7 @@
 %{_bindir}/tokuftdump
 %{_libdir}/mysql/plugin/ha_tokudb.so
 %attr(755, root, root) %{_libdir}/mysql/plugin/debug/ha_tokudb.so
+%attr(755, root, root) %{_bindir}/ps_tokudb_admin
 %endif
 
 # ----------------------------------------------------------------------------

=== modified file 'scripts/CMakeLists.txt'
--- scripts/CMakeLists.txt	2014-12-16 06:00:17 +0000
+++ scripts/CMakeLists.txt	2015-01-15 08:09:15 +0000
@@ -339,6 +339,7 @@
     mysqldumpslow
     mysqld_multi
     mysqld_safe
+    ps_tokudb_admin
   )
   FOREACH(file ${BIN_SCRIPTS})
     IF(EXISTS ${CMAKE_CURRENT_SOURCE_DIR}/${file}.sh)

=== modified file 'scripts/mysqld_safe.sh'
--- scripts/mysqld_safe.sh	2014-12-01 07:53:48 +0000
+++ scripts/mysqld_safe.sh	2015-01-15 08:09:15 +0000
@@ -20,6 +20,8 @@
 load_jemalloc=1
 flush_caches=0
 numa_interleave=0
+# Change (disable) transparent huge pages (TokuDB requirement)
+thp_setting=
 
 # Initial logging status: error log is not open, and not using syslog
 logging=init
@@ -233,6 +235,7 @@
       --open-files-limit=*) open_files="$val" ;;
       --open_files_limit=*) open_files="$val" ;;
       --skip-kill-mysqld*) KILL_MYSQLD=0 ;;
+      --thp-setting=*) thp_setting="$val" ;;
       --syslog) want_syslog=1 ;;
       --skip-syslog) want_syslog=0 ;;
       --syslog-tag=*) syslog_tag="$val" ;;
@@ -799,6 +802,45 @@
   exit 1
 fi
 
+# Change transparent huge pages setting if thp-setting option specified
+if [ -n "$thp_setting" ]
+then
+  if [ $(id -u) -ne 0 ]; then
+    log_error "mysqld_safe must be run as root for setting transparent huge pages!"
+    exit 1
+  elif [ $thp_setting != "always" -a $thp_setting != "madvise" -a $thp_setting != "never" ]; then
+    log_error "Invalid value for thp-setting=$thp_setting in config file. Valid values are: always, madvise or never"
+    exit 1
+  else
+    if [ -f /sys/kernel/mm/transparent_hugepage/enabled ]; then
+      CONTENT_THP=$(cat /sys/kernel/mm/transparent_hugepage/enabled)
+      STATUS_THP=0
+      set +e
+      STATUS_THP=$(echo $CONTENT_THP | grep -cv "\[${thp_setting}\]")
+      set -e
+    fi
+    if [ $STATUS_THP -eq 0 ]; then
+      log_notice "Transparent huge pages are already set to: ${thp_setting}."
+    else
+      if [ -f /sys/kernel/mm/transparent_hugepage/defrag ]; then
+        echo $thp_setting > /sys/kernel/mm/transparent_hugepage/defrag
+        if [ $? -ne 0 ]; then
+	  log_error "Error setting transparent huge pages to: ${thp_setting}."
+          exit 1
+        fi
+      fi
+      if [ -f /sys/kernel/mm/transparent_hugepage/enabled ]; then
+        echo $thp_setting > /sys/kernel/mm/transparent_hugepage/enabled
+        if [ $? -ne 0 ]; then
+	  log_error "Error setting transparent huge pages to: ${thp_setting}."
+          exit 1
+        fi
+      fi
+      log_notice "Successfuly set transparent huge pages to: ${thp_setting}."
+    fi
+  fi
+fi
+
 #
 # Uncomment the following lines if you want all tables to be automatically
 # checked and repaired during startup. You should add sensible key_buffer

=== added file 'scripts/ps_tokudb_admin.sh'
--- scripts/ps_tokudb_admin.sh	1970-01-01 00:00:00 +0000
+++ scripts/ps_tokudb_admin.sh	2015-01-15 08:09:15 +0000
@@ -0,0 +1,227 @@
+#!/bin/bash
+#
+# Script for installing TokuDB plugin in Percona Server
+#
+
+# Bail out on errors, be strict
+set -ue
+
+# Examine parameters
+# default user
+USER="root"
+# default pass
+PASSWORD=""
+STATUS_THP_SYSTEM=0
+STATUS_THP_MYCNF=0
+STATUS_PLUGIN=0
+ENABLE=0
+DISABLE=0
+MYCNF_LOCATION=
+MYSQLD_SAFE_STATUS=0
+
+# Check if we have a functional getopt(1)
+if ! getopt --test
+  then
+  go_out="$(getopt --options=u:p:edh \
+  --longoptions=user:,password:,enable,disable,help \
+  --name="$(basename "$0")" -- "$@")"
+  test $? -eq 0 || exit 1
+  eval set -- $go_out
+fi
+
+for arg
+do
+  case "$arg" in
+    -- ) shift; break;;
+    -u | --user )
+    shift
+    USER="$1"
+    ;;
+    -p | --password )
+    shift
+    PASSWORD="-p$1"
+    ;;
+    -e | --enable )
+    shift
+    ENABLE=1
+    ;;
+    -d | --disable )
+    shift
+    DISABLE=1
+    ;;
+    -h | --help )
+    printf "This script is used for installing and uninstalling TokuDB plugin for Percona Server 5.6.\n"
+    printf "If transparent huge pages are enabled on the system it adds thp-setting=never option to my.cnf\n"
+    printf "to disable it on runtime.\n\n"
+    printf "Valid options are:\n"
+    printf "  -u, --user <USER>\t mysql admin username\n"
+    printf "  -p, --password <PASS>\t mysql admin password\n"
+    printf "  -e, --enable\t\t enable TokuDB plugin and disable transparent huge pages in my.cnf\n"
+    printf "  -d, --disable\t\t disable TokuDB plugin and remove thp-setting=never option in my.cnf\n"
+    printf "  -h, --help\t\t show this help\n\n"
+    printf "For TokuDB requirements and manual steps for installation please visit this webpage:\n"
+    printf "http://www.percona.com/doc/percona-server/5.6/tokudb/tokudb_installation.html\n\n"
+    exit 0
+    ;;
+  esac
+done
+
+# Make sure only root can run this script
+if [ $(id -u) -ne 0 ]; then
+  echo "This script must be run as root!" 1>&2
+  exit 1
+fi
+
+if [ $ENABLE = 1 -a $DISABLE = 1 ]; then
+  printf "Only --enable OR --disable can be specified - not both!\n"
+  exit 1
+elif [ $ENABLE = 0 -a $DISABLE = 0 ]; then
+  printf "You should specify --enable or --disable option. Use --help for printing options.\n"
+  exit 1
+fi
+
+# Check transparent huge pages status on the system
+printf "Checking transparent huge pages status on the system...\n"
+if [ -f /sys/kernel/mm/transparent_hugepage/enabled ]; then
+  CONTENT_TRANSHP=$(</sys/kernel/mm/transparent_hugepage/enabled)
+  set +e
+  STATUS_THP_SYSTEM=$(echo $CONTENT_TRANSHP | grep -cv '\[never\]')
+  set -e
+fi
+if [ $STATUS_THP_SYSTEM = 0 ]; then
+  printf ">> Transparent huge pages are currently disabled on the system.\n\n"
+else
+  printf ">> Transparent huge pages are enabled (should be disabled).\n\n"
+fi
+
+# Check thp-setting=never option in my.cnf
+printf "Checking if thp-setting=never option is already set in config file...\n"
+set +e
+STATUS_THP_MYCNF=$(my_print_defaults server mysqld mysqld_safe|grep -c thp-setting=never)
+set -e
+if [ $STATUS_THP_MYCNF = 0 ]; then
+  printf ">> Option thp-setting=never is not set in the config file.\n"
+  printf ">> (needed only if THP is not disabled permanently on the system)\n\n"
+else
+  printf ">> Option thp-setting=never is set in the config file.\n\n"
+fi
+
+# Check location of my.cnf
+if [ -f /etc/my.cnf ]; then
+  MYCNF_LOCATION=/etc/my.cnf
+elif [ -f /etc/mysql/my.cnf ]; then
+  MYCNF_LOCATION=/etc/mysql/my.cnf
+elif [ -f /usr/etc/my.cnf ]; then
+  MYCNF_LOCATION=/usr/etc/my.cnf
+else
+  if [ -d /etc/mysql ]; then
+    MYCNF_LOCATION=/etc/mysql/my.cnf
+  else
+    MYCNF_LOCATION=/etc/my.cnf
+  fi
+  echo -n "" >> ${MYCNF_LOCATION}
+fi
+
+# Check TokuDB plugin status
+printf "Checking TokuDB plugin status...\n"
+LIST_ENGINE=$(mysql -e "show plugins;" -u$USER $PASSWORD)
+if [ $? -ne 0 ]; then
+  printf ">> Error checking TokuDB plugin status!\n";
+  exit 1
+fi
+set +e
+STATUS_PLUGIN=$(echo "$LIST_ENGINE" | grep -c "TokuDB")
+set -e
+if [ $STATUS_PLUGIN = 0 ]; then
+  printf ">> TokuDB plugin is not installed.\n\n"
+elif [ $STATUS_PLUGIN = 7 ]; then
+  printf ">> TokuDB plugin is installed.\n\n"
+else
+  printf ">> TokuDB plugin is partially installed. Please cleanup manually.\n\n"
+  exit 1
+fi
+
+# Disable transparent huge pages in the current session so
+# that the plugin can be installed without restarting PS
+if [ $ENABLE = 1 -a $STATUS_THP_SYSTEM = 1 ]; then
+  printf "Disabling transparent huge pages for the current session...\n"
+  if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
+    echo never > /sys/kernel/mm/transparent_hugepage/defrag
+  fi
+  if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
+    echo never > /sys/kernel/mm/transparent_hugepage/enabled
+  fi
+  if [ $? -eq 0 ]; then
+    printf ">> Successfuly disabled transparent huge pages for this session.\n\n"
+  else
+    printf ">> Error disabling transparent huge pages for this session.\n\n"
+    exit 1
+  fi
+fi
+
+# Add option to disable transparent huge pages into my.cnf
+if [ $ENABLE = 1 -a $STATUS_THP_MYCNF = 0 ]; then
+  printf "Adding thp-setting=never option into $MYCNF_LOCATION\n"
+  set +e
+  MYSQLD_SAFE_STATUS=$(grep -c "^\[mysqld_safe\]$" $MYCNF_LOCATION)
+  set -e
+  if [ $MYSQLD_SAFE_STATUS = 0 ]; then
+    echo -e "\n[mysqld_safe]\nthp-setting=never" >> $MYCNF_LOCATION
+  else
+    sed -i '/^\[mysqld_safe\]$/a thp-setting=never' $MYCNF_LOCATION
+  fi
+  if [ $? -eq 0 ]; then
+    printf ">> Successfuly added thp-setting=never option into $MYCNF_LOCATION\n\n";
+  else
+    printf ">> Error adding thp-setting=never option into $MYCNF_LOCATION\n\n";
+    exit 1
+  fi
+fi
+
+# Remove option for disabling transparent huge pages from my.cnf
+if [ $DISABLE = 1 -a $STATUS_THP_MYCNF = 1 ]; then
+  printf "Removing thp-setting=never option from $MYCNF_LOCATION\n"
+  sed -i '/^thp-setting=never$/d' $MYCNF_LOCATION
+  if [ $? -eq 0 ]; then
+    printf ">> Successfuly removed thp-setting=never option from $MYCNF_LOCATION\n\n";
+  else
+    printf ">> Error removing thp-setting=never option from $MYCNF_LOCATION\n\n";
+    exit 1
+  fi
+fi
+
+# Installing TokuDB plugin
+if [ $ENABLE = 1 -a $STATUS_PLUGIN = 0 ]; then
+  printf "Installing TokuDB engine...\n"
+  mysql -e "INSTALL PLUGIN tokudb SONAME 'ha_tokudb.so';" -u$USER $PASSWORD &&
+  mysql -e "INSTALL PLUGIN tokudb_file_map SONAME 'ha_tokudb.so';" -u$USER $PASSWORD &&
+  mysql -e "INSTALL PLUGIN tokudb_fractal_tree_info SONAME 'ha_tokudb.so';" -u$USER $PASSWORD &&
+  mysql -e "INSTALL PLUGIN tokudb_fractal_tree_block_map SONAME 'ha_tokudb.so';" -u$USER $PASSWORD &&
+  mysql -e "INSTALL PLUGIN tokudb_trx SONAME 'ha_tokudb.so';" -u$USER $PASSWORD &&
+  mysql -e "INSTALL PLUGIN tokudb_locks SONAME 'ha_tokudb.so';" -u$USER $PASSWORD &&
+  mysql -e "INSTALL PLUGIN tokudb_lock_waits SONAME 'ha_tokudb.so';" -u$USER $PASSWORD
+  if [ $? -eq 0 ]; then
+    printf ">> Successfuly installed TokuDB plugin.\n\n"
+  else
+    printf ">> Error installing TokuDB plugin. Please check error log.\n\n"
+    exit 1
+  fi
+fi
+
+# Uninstalling TokuDB plugin
+if [ $DISABLE = 1 -a $STATUS_PLUGIN = 7 ]; then
+  printf "Uninstalling TokuDB plugin...\n"
+  mysql -e "UNINSTALL PLUGIN tokudb;" -u$USER $PASSWORD &&
+  mysql -e "UNINSTALL PLUGIN tokudb_file_map;" -u$USER $PASSWORD &&
+  mysql -e "UNINSTALL PLUGIN tokudb_fractal_tree_info;" -u$USER $PASSWORD &&
+  mysql -e "UNINSTALL PLUGIN tokudb_fractal_tree_block_map;" -u$USER $PASSWORD &&
+  mysql -e "UNINSTALL PLUGIN tokudb_trx;" -u$USER $PASSWORD &&
+  mysql -e "UNINSTALL PLUGIN tokudb_locks;" -u$USER $PASSWORD &&
+  mysql -e "UNINSTALL PLUGIN tokudb_lock_waits;" -u$USER $PASSWORD
+  if [ $? -eq 0 ]; then
+    printf ">> Successfuly uninstalled TokuDB plugin.\n\n"
+  else
+    printf ">> Error uninstalling TokuDB plugin. Please check error log.\n\n"
+    exit 1
+  fi
+fi