Merge lp:~robert-ayres/charms/precise/hive/mysql-update into lp:charms/hive

Proposed by Robert Ayres
Status: Merged
Merged at revision: 6
Proposed branch: lp:~robert-ayres/charms/precise/hive/mysql-update
Merge into: lp:charms/hive
Diff against target: 96 lines (+6/-44)
2 files modified
README.md (+2/-43)
hooks/hive-common (+4/-1)
To merge this branch: bzr merge lp:~robert-ayres/charms/precise/hive/mysql-update
Reviewer Review Type Date Requested Status
Marco Ceppi (community) Approve
Review via email: mp+200577@code.launchpad.net

Description of the change

Updates to current MySQL charm removing the previous Known Issues and workarounds.

To post a comment you must log in.
Revision history for this message
Marco Ceppi (marcoceppi) wrote :

LGTM +1

review: Approve

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
=== modified file 'README.md'
--- README.md 2013-09-09 14:38:23 +0000
+++ README.md 2014-01-06 20:20:46 +0000
@@ -51,8 +51,8 @@
51A Hive server stores metadata in MySQL::51A Hive server stores metadata in MySQL::
5252
53 juju deploy mysql53 juju deploy mysql
5454 # hive requires ROW binlog
55**Please read Known Issues 1. below before proceeding further**55 juju set mysql binlog-format=ROW
5656
57To deploy a Hive service without a Metastore service::57To deploy a Hive service without a Metastore service::
5858
@@ -61,8 +61,6 @@
61 # associate Hive with MySQL61 # associate Hive with MySQL
62 juju add-relation hive-server:db mysql:db62 juju add-relation hive-server:db mysql:db
6363
64**Please read Known Issues 2. below before proceeding further**
65
66 # associate Hive with HDFS Namenode64 # associate Hive with HDFS Namenode
67 juju add-relation hive-server:namenode hadoop-master:namenode65 juju add-relation hive-server:namenode hadoop-master:namenode
68 # associate Hive with Jobtracker66 # associate Hive with Jobtracker
@@ -75,8 +73,6 @@
75 # associate Metastore with MySQL73 # associate Metastore with MySQL
76 juju add-relation hive-metastore:db mysql:db74 juju add-relation hive-metastore:db mysql:db
7775
78**Please read Known Issues 2. below before proceeding further**
79
80 # associate Metastore with Namenode76 # associate Metastore with Namenode
81 juju add-relation hive-metastore:namenode hadoop-master:namenode77 juju add-relation hive-metastore:namenode hadoop-master:namenode
8278
@@ -94,40 +90,3 @@
94 juju add-unit hive-server90 juju add-unit hive-server
9591
96This currently only works when using a Metastore service.92This currently only works when using a Metastore service.
97
98## Known Issues
99
100
101 1. The current MySQL Charm does not alter the binlog format to 'ROW' (nor can
102the binlog be disabled) so attempting to use Hive will result in a nasty error
103message of::
104
105 'Cannot execute statement: impossible to write to binary log since
106 BINLOG_FORMAT = STATEMENT and at least one table uses a storage engine
107 limited to row-based logging. InnoDB is limited to row-logging when
108 transaction isolation level is READ COMMITTED or READ UNCOMMITTED.'
109
110The current solution is to ssh into a MySQL charm machine and manually set the
111binlog format::
112
113 sudo bash -c 'echo "binlog_format = ROW" >> /etc/mysql/conf.d/binlog.cnf'
114 sudo service mysql restart
115
116This should be done after deploying the MySQL charm.
117
118 1. The current MySQL Charm creates a database using 'utf8' encoding, which
119will give you the following error with Hive::
120
121 'MetaException(message:Got exception: javax.jdo.JDODataStoreException An
122 exception was thrown while adding/validating class(es) : Specified key was
123 too long; max key length is 767 bytes'
124
125The current solution is drop the database and recreate it (to use 'latin1'
126encoding)::
127
128 mysql -u root -p`cat /var/lib/juju/mysql.passwd` -e 'drop database `<name>`; \
129 create database `<name>`;'
130
131where '<name>' is the service name, either 'hive-server' or 'hive-metastore'.
132
133This should be done after you have associated Hive with the database.
13493
=== modified file 'hooks/hive-common'
--- hooks/hive-common 2012-03-22 15:33:35 +0000
+++ hooks/hive-common 2014-01-06 20:20:46 +0000
@@ -186,7 +186,7 @@
186186
187install_mysql_jdbc () {187install_mysql_jdbc () {
188 juju-log "Installing mysql jdbc driver"188 juju-log "Installing mysql jdbc driver"
189 apt-get -y install libmysql-java189 apt-get -y install libmysql-java mysql-client
190}190}
191191
192configure_mysql () {192configure_mysql () {
@@ -195,6 +195,9 @@
195 database=`relation-get database`195 database=`relation-get database`
196 user=`relation-get user`196 user=`relation-get user`
197 password=`relation-get password`197 password=`relation-get password`
198 # mysql charm creates databases defaulting to utf-8, hive requires latin1
199 mysql -h $host -u "$user" "-p$password" "$database" \
200 -e "ALTER DATABASE \`$database\` CHARACTER SET=latin1"
198 dir=`dotdee --dir /etc/hive/conf.juju/hive-site.xml`201 dir=`dotdee --dir /etc/hive/conf.juju/hive-site.xml`
199 rm -f $dir/6*-* $dir/7*-*202 rm -f $dir/6*-* $dir/7*-*
200 config_element hive.metastore.local true > $dir/61-hive.metastore.local203 config_element hive.metastore.local true > $dir/61-hive.metastore.local

Subscribers

People subscribed via source and target branches

to all changes: