MAAS

Commissioning script 'maas-kernel-cmdline' fails with bonded interfaces

Bug #1985741 reported by Alan Baghumian
26
This bug affects 5 people
Affects Status Importance Assigned to Milestone
MAAS
Fix Released
High
Adam Collard
MAAS 3.3.0-beta1

Bug Description

If an existing MAAS node is configured to use bonding or bridging + bonding, re-commissioning attempts fail at the "maas-kernel-cmdline" step if "Retain Network Configuration" is selected.

Inspecting region controller logs show the following trace:

2022-08-11 19:27:06 metadataserver.api: [critical] os-node-1.alannet(wp8t7q): commissioning script 'maas-kernel-cmdline' failed during post-processing.
    Traceback (most recent call last):
      File "/snap/maas/23178/lib/python3.8/site-packages/metadataserver/api.py", line 866, in signal
        target_status = process(node, request, status)
      File "/snap/maas/23178/lib/python3.8/site-packages/metadataserver/api.py", line 690, in _process_commissioning
        self._store_results(
      File "/snap/maas/23178/lib/python3.8/site-packages/metadataserver/api.py", line 575, in _store_results
        script_result.store_result(
      File "/snap/maas/23178/lib/python3.8/site-packages/metadataserver/models/scriptresult.py", line 372, in store_result
        signal_status = try_or_log_event(
    --- <exception caught here> ---
      File "/snap/maas/23178/lib/python3.8/site-packages/metadataserver/api.py", line 493, in try_or_log_event
        func(*args, **kwargs)
      File "/snap/maas/23178/lib/python3.8/site-packages/metadataserver/builtin_scripts/hooks.py", line 207, in update_boot_interface
        node.boot_interface = node.current_config.interface_set.get(
      File "/snap/maas/23178/usr/lib/python3/dist-packages/django/db/models/manager.py", line 82, in manager_method
        return getattr(self.get_queryset(), name)(*args, **kwargs)
      File "/snap/maas/23178/usr/lib/python3/dist-packages/django/db/models/query.py", line 410, in get
        raise self.model.MultipleObjectsReturned(
    maasserver.models.interface.MultipleObjectsReturned: get() returned more than one Interface -- it returned 2!

You can re-commission the machine by unchecking "Retain Network Configuration" however that obviously wipes the network configuration.

Steps to reproduce:

1. Commission a multi-interface machine in MAAS.
2. Configure the network to use bonding.
3. Attempt to re-commission the machine.
4. The attempt fails at the "maas-kernel-cmdline" step.

Tested Version: 3.2.2-12005-g.f0f96fa78 (Snap).

Attached a sample network configuration.

Thank you.

See original description

Related branches

~adam-collard/maas:boot-interface-physical-interface-only
MAAS Lander: Needs Fixing
Alberto Donato (community): Approve
Diff: 147 lines (+59/-32)
2 files modified
src/metadataserver/builtin_scripts/hooks.py (+6/-2)
src/metadataserver/builtin_scripts/tests/test_hooks.py (+53/-30)
Revision history for this message
Alan Baghumian (alanbach) wrote :
description: updated
Alexsander de Souza (alexsander-souza)
Changed in maas:
status: New → Triaged
importance: Undecided → High
milestone: none → 3.3.0
Revision history for this message
Cooper Hopkin (chopkin) wrote :
Download full text (8.2 KiB)

I have the same problem, and I can confirm it's very new. I was able to commission a node with an OpenVswitch bridge as late as 2022-08-10, but ran into this bug when trying to commission a new machine on 2022-08-15. Same error in the controller log. I found something interesting though. In regiond.log, it looks like maas started downloading a new kernel release ['20220810']. It seems to be the only thing that changed between 2022-08-10 and 2022-08-11. This is the first time it shows up in the logs:

2022-08-11 20:50:55 sstreams: [info] com.ubuntu.maas:stable:v3:download/com.ubuntu.maas.stable:v3:boot:18.04:amd64:ga-18.04: to_add=['20220810'] to_remove=[]
2022-08-11 20:50:55 sstreams: [info] com.ubuntu.maas:stable:v3:download/com.ubuntu.maas.stable:v3:boot:18.04:amd64:ga-18.04-lowlatency: to_add=['20220810'] to_remove=[]
2022-08-11 20:50:55 sstreams: [info] com.ubuntu.maas:stable:v3:download/com.ubuntu.maas.stable:v3:boot:18.04:amd64:hwe-18.04: to_add=['20220810'] to_remove=[]
2022-08-11 20:50:55 sstreams: [info] com.ubuntu.maas:stable:v3:download/com.ubuntu.maas.stable:v3:boot:18.04:amd64:hwe-18.04-edge: to_add=['20220810'] to_remove=[]
2022-08-11 20:50:55 sstreams: [info] com.ubuntu.maas:stable:v3:download/com.ubuntu.maas.stable:v3:boot:18.04:amd64:hwe-18.04-lowlatency: to_add=['20220810'] to_remove=[]
2022-08-11 20:50:55 sstreams: [info] com.ubuntu.maas:stable:v3:download/com.ubuntu.maas.stable:v3:boot:18.04:amd64:hwe-18.04-lowlatency-edge: to_add=['20220810'] to_remove=[]
2022-08-11 20:50:55 sstreams: [info] com.ubuntu.maas:stable:v3:download/com.ubuntu.maas.stable:v3:boot:18.04:arm64:ga-18.04: to_add=['20220810'] to_remove=[]
2022-08-11 20:50:55 sstreams: [info] com.ubuntu.maas:stable:v3:download/com.ubuntu.maas.stable:v3:boot:18.04:arm64:hwe-18.04: to_add=['20220810'] to_remove=[]
2022-08-11 20:50:55 sstreams: [info] com.ubuntu.maas:stable:v3:download/com.ubuntu.maas.stable:v3:boot:18.04:arm64:hwe-18.04-edge: to_add=['20220810'] to_remove=[]
2022-08-11 20:50:55 sstreams: [info] com.ubuntu.maas:stable:v3:download/com.ubuntu.maas.stable:v3:boot:18.04:arm64:xgene-uboot: to_add=['20220810'] to_remove=[]
2022-08-11 20:50:55 sstreams: [info] com.ubuntu.maas:stable:v3:download/com.ubuntu.maas.stable:v3:boot:18.04:arm64:xgene-uboot-mustang: to_add=['20220810'] to_remove=[]
2022-08-11 20:50:55 sstreams: [info] com.ubuntu.maas:stable:v3:download/com.ubuntu.maas.stable:v3:boot:18.04:armhf:ga-18.04: to_add=['20220810'] to_remove=[]
2022-08-11 20:50:55 sstreams: [info] com.ubuntu.maas:stable:v3:download/com.ubuntu.maas.stable:v3:boot:18.04:armhf:generic-lpae-generic-lpae: to_add=['20220810'] to_remove=[]
2022-08-11 20:50:55 sstreams: [info] com.ubuntu.maas:stable:v3:download/com.ubuntu.maas.stable:v3:boot:18.04:armhf:hwe-18.04: to_add=['20220810'] to_remove=[]
2022-08-11 20:50:55 sstreams: [info] com.ubuntu.maas:stable:v3:download/com.ubuntu.maas.stable:v3:boot:18.04:armhf:hwe-18.04-edge: to_add=['20220810'] to_remove=[]
2022-08-11 20:50:55 sstreams: [info] com.ubuntu.maas:stable:v3:download/com.ubuntu.maas.stable:v3:boot:18.04:i386:ga-18.04: to_add=['20220810'] to_remove=[]
2022-08-11 20:50:55 sstreams: [info] com.ubuntu.maas:stable:v3:download/com...

Read more...

Revision history for this message
Björn Tillenius (bjornt) wrote :

I think the fix here is to filter on PHYSICAL interfaces.

Adam Collard (adam-collard)
Changed in maas:
assignee: nobody → Adam Collard (adam-collard)
MAAS Lander (maas-lander)
Changed in maas:
status: Triaged → Fix Committed
Alexsander de Souza (alexsander-souza)
Changed in maas:
milestone: 3.3.0 → 3.3.0-beta1
Alexsander de Souza (alexsander-souza)
Changed in maas:
status: Fix Committed → Fix Released
Peter Matulis (petermatulis)
no longer affects: charm-deployment-guide
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.