Access denied for user debian-sys-maint

Bug #1098360 reported by Henrik Ingo
18
This bug affects 3 people
Affects Status Importance Assigned to Milestone
Percona Server moved to https://jira.percona.com/projects/PS
Fix Released
Low
Hrvoje Matijakovic
5.1
Fix Released
Low
Hrvoje Matijakovic
5.5
Fix Released
Low
Hrvoje Matijakovic
5.6
Fix Released
Low
Hrvoje Matijakovic
Percona XtraDB Cluster moved to https://jira.percona.com/projects/PXC
Fix Released
Undecided
Raghavendra D Prabhu

Bug Description

I get the following error when starting mysqld from latest PXC 5.5 on Ubuntu Precise.

ERROR 1045 (28000): Access denied for user 'debian-sys-maint'@'localhost' (using password: YES)

For this reason, stopping MySQL also fails:

/etc/init.d/mysql stop
 * Stopping MySQL (Percona XtraDB Cluster) mysqld [fail]

Debian MySQL packages create their own debian-sys-maint user for administrative tasks. The password is generated randomly by dpkg install scripts and stored in /etc/mysql/debian.cnf. When joining a Galera cluster we will of course copy mysql.user table from the donor node, where the same user has a different password. This leads to the above errors.

I was using mysqldump sst. I can imagine the same happens with other sst methods too.

To fix this, it would be necessary to copy the debian.cnf file from the donor. Alternatively the sst script would have to somehow regenerate a new password for debian-sys-maint user at the completion of sst.

I did not check if Codership deb packages have the same issue.

Related branches

Revision history for this message
Henrik Ingo (hingo) wrote :

More info, via Alex: Codership deb packages use a different init script, which doesn't use the debian.cnf file in any way. You should probably copy that from upstream and the problem goes away.

tags: added: init-script
Revision history for this message
Raghavendra D Prabhu (raghavendra-prabhu) wrote :

Yes, using a different init script should do or running post-install configure after installation as mentioned in lp:1099429

Changed in percona-xtradb-cluster:
status: New → Confirmed
Alexey Bychko (abychko)
tags: added: pkg
Revision history for this message
Raghavendra D Prabhu (raghavendra-prabhu) wrote :

I checked the init script in galera repo and it is still

MYADMIN="/usr/bin/mysqladmin --defaults-file=/etc/mysql/debian.cnf"

Now, since this is a cluster environment, it doesn't make sense
to have different passwords for different boxes.

However, not providing the config file imples that, like
RHEL/CentOS, the root password is empty or provided in the cnf
file, otherwise the init will fail again.

Another possibility is to ignore the replication of mysql.users
table and run dpkg reconfigure after every SST. But this can get
complicated.

Revision history for this message
Jay Janssen (jay-janssen) wrote :

"Another possibility is to ignore the replication of mysql.users
table and run dpkg reconfigure after every SST. But this can get
complicated."

Whatever you do, don't do that. I'd rather see Debian's init script get neutered.

Changed in percona-xtradb-cluster:
milestone: none → 5.5.30-24.8
Revision history for this message
Raghavendra D Prabhu (raghavendra-prabhu) wrote :

The CentOS script uses kill signals than use mysqladmin, which should be
fine.

Also, as discussed, this requires changes in PS's debian init script
too, for consistency.

Revision history for this message
Raghavendra D Prabhu (raghavendra-prabhu) wrote :

Note, in PS's case, since there is no SST or any such automated
self-deployment, it is not that severe, but in cases where a
backup of say, master, is restored on slave, it can fail similarly.

Changed in percona-xtradb-cluster:
status: Confirmed → Fix Committed
assignee: nobody → Raghavendra D Prabhu (raghavendra-prabhu)
Revision history for this message
Valerii Kravchuk (valerii-kravchuk) wrote :

For PS we should either document steps to fix this or, even better, probably just replace/patch/fix init scripts when PS is installed.

Revision history for this message
Laurynas Biveinis (laurynas-biveinis) wrote :

It seems to be not that critical in a non-cluster environment, I'm not familiar with how Debian manages MySQL, but would leaving the current behavior be least surprising there? Maybe let's just document it for PS for now?

Revision history for this message
Valerii Kravchuk (valerii-kravchuk) wrote :

If we want PS to be a drop-in replacement for Debian's provided MySQL, then probably we should not change the way init scripts work. Maybe we should just add some notes to the manual that after restoring mysql database from backup created on other server one has to deal with the problem outlined here somehow.

tags: added: doc
Revision history for this message
Przemek (pmalkowski) wrote :

IMHO for PS not changing Debian's init scripts can make sense, but XtraDB Cluster is a much different product.
So after a new node joins the cluster for the first time, it's /etc/mysql/debian.cnf is no longer valid.

Also the upgrade_system_tables_if_necessary and check_for_crashed_tables parts of debian-start makes little to no sense on PXC.
The upgrade_system_tables_if_necessary will execute mysql_upgrade each time it won't see the mysql_upgrade_info file in place. So if an existing node is re-created with full SST, it won't get the mysql_upgrade_info file copied over with xtrabackup. The effect is that this node will work fine until you restart it. On restart, debian-start will notice there is no mysql_upgrade_info present, so it will fire mysql_upgrade, which does all the ALTERs for tables in mysql database whether those need an upgrade or not. And even though such ALTER fails on the node it's run on, still will be replicated via wsrep to other nodes making GRA files and errors on them. Just try doing "ALTER TABLE test.foo engine=innodb" on any node.

Changed in percona-xtradb-cluster:
status: Fix Committed → Fix Released
Revision history for this message
Frank Papenmeier (frank-papenmeier) wrote :

I wonder how this issue has been fixed? I did a clean install on two wheezy machines today and ran into this very same issue after the SST when adding the node (Access denied for user 'debian-sys-maint'@'localhost')? What should the debian startup script look like? Thanks!

Revision history for this message
Shahriyar Rzayev (rzayev-sehriyar) wrote :

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PXC-1276

Revision history for this message
Shahriyar Rzayev (rzayev-sehriyar) wrote :

Percona now uses JIRA for bug reports so this bug report is migrated to: https://jira.percona.com/browse/PS-1962

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.