[tags] Unable to deploy cluster with new role clonned from controller role

Bug #1669743 reported by Vladimir Khlyunev
6
This bug affects 1 person
Affects Status Importance Assigned to Milestone
Fuel for OpenStack
Fix Committed
High
Georgy Kibardin
Newton
Fix Released
High
Georgy Kibardin

Bug Description

ISO 1423

Steps to reproduce:
- "Clone" controller role to new role
- Deploy cluster with 1 new clonned controller role

Result - deployment fails with:
2017-03-02 22:03:21 +0000 Puppet (err): Could not prefetch mysql_database provider 'mysql': Execution of '/usr/bin/mysql --defaults-extra-file=/root/.my.cnf -NBe show databases' returned 1: ERROR 2003 (HY000): Can't connect to MySQL server on '10.109.11.2' (111)

And other connection errros

snapshot https://product-ci.infra.mirantis.net/job/10.0.system_test.ubuntu.test_tags/18/artifact/logs/fail_error_roles_to_tags-fuel-snapshot-2017-03-02_22-03-40.tar

tags: added: swarm-fail
Changed in fuel:
status: New → Confirmed
Revision history for this message
Oleksiy Molchanov (omolchanov) wrote :

root@node-1:~# mysql --protocol=SOCKET
ERROR 2047 (HY000): Wrong or unknown protocol

root@node-1:~# dpkg -l | grep mysql
ii libdbd-mysql-perl 4.033-1ubuntu0.1 amd64 Perl5 database interface to the MySQL database
ii libmysqlclient20:amd64 5.7.17-0ubuntu0.16.04.1 amd64 MySQL database client library
ii mysql-common 5.7.17-0ubuntu0.16.04.1 all MySQL database common files, e.g. /etc/mysql/my.cnf
ii mysql-wsrep-client-5.6 5.6.35-0~u16.04+mos1 amd64 MySQL database client binaries
ii mysql-wsrep-common-5.6 5.6.35-0~u16.04+mos1 all MySQL database common files, e.g. /etc/mysql/conf.d/my-5.6.cnf
ii mysql-wsrep-libmysqlclient18:amd64 5.6.35-0~u16.04+mos1 amd64 MySQL database client library
ii mysql-wsrep-server-5.6 5.6.35-0~u16.04+mos1 amd64 MySQL database wsrep server binaries and system database setup

Changed in fuel:
assignee: Fuel Sustaining (fuel-sustaining-team) → MOS Linux (mos-linux)
importance: Undecided → Critical
tags: added: area-linux
Revision history for this message
Ivan Suzdal (isuzdal) wrote :

Seems like some issue with host settings in .my.cnf. Please, set it to localhost.

Changed in fuel:
assignee: MOS Linux (mos-linux) → Oleksiy Molchanov (omolchanov)
tags: added: blocker-for-qa
Revision history for this message
Oleksiy Molchanov (omolchanov) wrote :

After further investigation we found out that in this env haproxy is not deployed, because there is no haproxy_vip.

<% # If we have no VIPs with 'haproxy' namespace, then we're using external LB
if @vips.select{|key, hash| hash['namespace'] == 'haproxy' }.length == 0
 globals.store "external_lb", true
else
 globals.store "external_lb", false
end -%>

https://github.com/openstack/fuel-qa/blob/stable/newton/fuelweb_test/tests/tests_lcm/test_tags.py#L246-L321

Changed in fuel:
assignee: Oleksiy Molchanov (omolchanov) → Fuel Sustaining (fuel-sustaining-team)
tags: added: area-python
Revision history for this message
Oleksiy Molchanov (omolchanov) wrote :

Actually there is no vips in network metadata

Dmitry Pyzhov (dpyzhov)
Changed in fuel:
assignee: Fuel Sustaining (fuel-sustaining-team) → Georgy Kibardin (gkibardin)
Revision history for this message
Georgy Kibardin (gkibardin) wrote :

[root@nailgun ~]# fuel nodes
id | status | name | cluster | ip | mac | roles | pending_roles | online | group_id
---+--------+-------------------------+---------+-------------+-------------------+----------------+---------------+--------+---------
 1 | error | slave-01_controller_new | 1 | 10.109.10.4 | 64:2a:3e:e5:a3:88 | controller_new | | 1 | 1

controller_new - looks pretty strange.

Revision history for this message
Georgy Kibardin (gkibardin) wrote :

Here is where it is from:
2017-03-14 21:46:28.309 DEBUG [7f6e5d038880] (logger) Request POST /api/releases/2/roles from 10.109.10.1:59052 {"meta": {"group": "base", "description": "New controller role", "weight": 10, "tags": ["controller", "database", "keystone", "neutron", "new_tag"], "conflicts": ["compute"], "public_for_dvr_required": true, "name": "Controller_new", "update_required": ["compute", "cinder"], "public_ip_required": true}, "name": "controller_new", "volumes_roles_mapping": [{"id": "os", "allocate_size": "min"}, {"id": "logs", "allocate_size": "min"}, {"id": "image", "allocate_size": "all"}, {"id": "mysql", "allocate_size": "min"}, {"id": "horizon", "allocate_size": "min"}]}
2017-03-14 21:46:28.360 DEBUG [7f6e5d038880] (logger) Response code '201 Created' for POST /api/releases/2/roles from 10.109.10.1:59052

Revision history for this message
Georgy Kibardin (gkibardin) wrote :
Changed in fuel:
assignee: Georgy Kibardin (gkibardin) → Fuel QA Team (fuel-qa)
description: updated
summary: - [tags] Unable to deploy controller with removed tag 'rabbitmq'
+ [tags] Unable to deploy cluster with new role clonned from controller
+ role
Revision history for this message
Georgy Kibardin (gkibardin) wrote :

The reason is that
https://github.com/openstack/fuel-web/blob/master/nailgun/nailgun/objects/cluster.py#L1009
uses hardcoded role name rather than tags.

Changed in fuel:
assignee: Fuel QA Team (fuel-qa) → Fuel Sustaining (fuel-sustaining-team)
importance: Critical → Medium
Revision history for this message
Georgy Kibardin (gkibardin) wrote :

So, by now the workaround is "do not rename controller role" :)

Changed in fuel:
importance: Medium → High
Changed in fuel:
assignee: Fuel Sustaining (fuel-sustaining-team) → Georgy Kibardin (gkibardin)
status: Confirmed → In Progress
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (master)

Fix proposed to branch: master
Review: https://review.openstack.org/447417

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-web (master)

Reviewed: https://review.openstack.org/447417
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=fba3bc344523da0d43f94e130b4944efe0d32529
Submitter: Jenkins
Branch: master

commit fba3bc344523da0d43f94e130b4944efe0d32529
Author: Georgy Kibardin <email address hidden>
Date: Mon Mar 20 11:59:21 2017 +0300

    Use tags instead of roles

    Now the deployment behaviour is driven by tags rather than role names, i.e. role
    name cannot be relied upon.

    Change-Id: Icfabeeb0b7fb6a9d697a09c3cf1fa020bbd4c323
    Closes-Bug: #1669743

Changed in fuel:
status: In Progress → Fix Committed
Dmitry Pyzhov (dpyzhov)
Changed in fuel:
milestone: 10.1 → 12.0
Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix proposed to fuel-web (stable/newton)

Fix proposed to branch: stable/newton
Review: https://review.openstack.org/449594

Revision history for this message
OpenStack Infra (hudson-openstack) wrote : Fix merged to fuel-web (stable/newton)

Reviewed: https://review.openstack.org/449594
Committed: https://git.openstack.org/cgit/openstack/fuel-web/commit/?id=72eaa18921a5dce874eeca0003b100311a552f50
Submitter: Jenkins
Branch: stable/newton

commit 72eaa18921a5dce874eeca0003b100311a552f50
Author: Georgy Kibardin <email address hidden>
Date: Mon Mar 20 11:59:21 2017 +0300

    Use tags instead of roles

    Now the deployment behaviour is driven by tags rather than role names, i.e. role
    name cannot be relied upon.

    Change-Id: Icfabeeb0b7fb6a9d697a09c3cf1fa020bbd4c323
    Closes-Bug: #1669743
    (cherry picked from commit fba3bc344523da0d43f94e130b4944efe0d32529)

Revision history for this message
Dmitry Belyaninov (dbelyaninov) wrote :
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.