MAAS

[1.9] If rack and region have different versions, the error is uninformative and confusing

Bug #1567249 reported by Haw Loeung
16
This bug affects 3 people
Affects Status Importance Assigned to Milestone
MAAS
Invalid
Undecided
Unassigned
1.9
Fix Released
Undecided
Unassigned
MAAS 1.9.4

Bug Description

Hi,

We've just recently upgraded our MAAS cluster (one region controller and six cluster controllers) from 1.7 to 1.9. After upgrading, we ran into the following traceback both in the logs and also when trying to view a node:

| File "/usr/lib/python2.7/dist-packages/maasserver/websockets/base.py", line 205, in full_dehydrate
| return self.dehydrate(obj, data, for_list=for_list)
| File "/usr/lib/python2.7/dist-packages/maasserver/websockets/handlers/cluster.py", line 59, in dehydrate
| data["power_types"] = self.dehydrate_power_types(obj)
| File "/usr/lib/python2.7/dist-packages/maasserver/websockets/handlers/cluster.py", line 65, in dehydrate_power_types
| return get_all_power_types_from_clusters(nodegroups=[obj])
| File "/usr/lib/python2.7/dist-packages/maasserver/clusterrpc/power_parameters.py", line 202, in get_all_power_types_from_clusters
| missing_packages = power_type['missing_packages']
| exceptions.KeyError: u'missing_packages'

I worked around this by changing the code in maasserver/clusterrpc/power_parameters.py to:

| description = power_type['description']
| if 'missing_packages' in power_type:
| missing_packages = power_type['missing_packages']
| else:
| missing_packages = None

Is that the correct fix for this?

Related branches

lp:~hloeung/maas/no-missing-packages
lp:~lamont/maas/bug-1567249-1.9
Blake Rouse (community): Approve
Diff: 79 lines (+35/-1)
3 files modified
docs/changelog.rst (+2/-0)
src/maasserver/clusterrpc/power_parameters.py (+15/-1)
src/maasserver/clusterrpc/tests/test_power_parameters.py (+18/-0)
Revision history for this message
Blake Rouse (blake-rouse) wrote :

Are you sure that all cluster controllers have the same installed version of MAAS as the region controller? MAAS requires that all of them be the same version.

Changed in maas:
status: New → Incomplete
Revision history for this message
Haw Loeung (hloeung) wrote :

@blake-rouse, ah that would appear it. I had later realised I missed one cluster controller and upgraded that.

Anyways, I reverted the change (and bounced regiond) and it appears to be working fine now - no tracebacks.

Still, I think an error about possible mismatch in versions or something else would be nicer as it took a bit of time for me figuring out how to get our MAAS cluster back up and running.

Revision history for this message
Launchpad Janitor (janitor) wrote :

[Expired for MAAS because there has been no activity for 60 days.]

Changed in maas:
status: Incomplete → Expired
LaMont Jones (lamont)
summary: - 'missing_packages' missing and causes traceback
+ 'missing_packages' missing and causes traceback on version mismatches
summary: - 'missing_packages' missing and causes traceback on version mismatches
+ If rack and region have different versions, the error is uninformative
+ and confusing
Changed in maas:
status: Expired → Confirmed
milestone: none → 1.9.4
Revision history for this message
LaMont Jones (lamont) wrote : Re: If rack and region have different versions, the error is uninformative and confusing

Based on the specific error we hit above, we should be able to make MAAS provide a clearer error message for that KeyError, since it indicates that an older version of MAAS is installed on one of the cluster controllers. We should indicate which cluster controller in the error message, if we can determine that.

Revision history for this message
LaMont Jones (lamont) wrote :

If the region is running 2.0 or 2.1, then the 1.7 cluster simply fails to register, and the region log shows it connecting and disconnecting, over and over and over. While not ideal, the admin will likely go look into why the cluster isn't there, and notice that it's downrev.

LaMont Jones (lamont)
summary: - If rack and region have different versions, the error is uninformative
- and confusing
+ [1.9] If rack and region have different versions, the error is
+ uninformative and confusing
LaMont Jones (lamont)
no longer affects: maas/2.0
no longer affects: maas/trunk
Andres Rodriguez (andreserl)
Changed in maas:
milestone: 2.0.1 → none
status: Confirmed → Incomplete
Revision history for this message
Andres Rodriguez (andreserl) wrote :

Hi!

**This is an automated message**

We believe this is may no longer be an issue in the latest MAAS release. Due to the report date of this, we are currently marking it as Invalid. If you believe this bug report still valid against the latest release of MAAS, or if you are still interested in this, please re-open this bug report.

Thanks

Changed in maas:
status: Incomplete → Invalid
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.