Deploying node with di on armhf/keystone can't find BOOTIF

Bug #1350302 reported by Newell Jensen
18
This bug affects 2 people
Affects Status Importance Assigned to Milestone
MAAS
Fix Released
Critical
Unassigned
netcfg (Ubuntu)
Fix Released
High
Colin Watson
Trusty
Fix Released
High
Colin Watson

Bug Description

[Impact]
The interface=auto preseed option doesn't work on (some?) u-boot based systems, making d-i and MAAS fail in configurations where multiple external network interfaces are present.

[Test Case]
1) Get a u-boot-based system with multiple NICs attached.
2) pxe boot d-i from the second NIC, passing "IPAPPEND 2" in the pxelinux config file
3) Observe the warning "netcfg[$pid]: INFO: Could not find valid BOOTIF= entry in /proc/cmdline" in /var/log/syslog.

When it is working, the installer should automatically try and configure whatever NIC was used to pxeboot, and the above error should not appear in syslog.

[Regression Risk]
The regression risk I see is that the new BOOTIF parsing code somehow regresses this feature on non-u-boot based systems.

Changed in maas:
importance: High → Critical
Colin Watson (cjwatson)
affects: debian-installer → debian-installer (Ubuntu)
affects: debian-installer (Ubuntu) → netcfg (Ubuntu)
Changed in maas:
status: New → Triaged
milestone: 1.5.4 → none
Revision history for this message
fish (discordianfish) wrote :

Same issue here with plain initrd built by initramfs on trusty, so it looks like a netcfg issue. Don't know how to figure out the version of netcfg, but initramfs-tools version is 0.103ubuntu4.2.

Revision history for this message
fish (discordianfish) wrote :

Sorry, sent it out too soon: This is happening here on a regular x86_64, so doesn't seem to be specific to armhf/keystone.

I dropped to a shell as well and verified that BOOTIF is set to the correct mac address.

Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in netcfg (Ubuntu):
status: New → Confirmed
Revision history for this message
Newell Jensen (newell-jensen) wrote :

fish, any progress on this at all?

Colin Watson (cjwatson)
Changed in netcfg (Ubuntu):
assignee: nobody → Colin Watson (cjwatson)
importance: Undecided → High
status: Confirmed → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package netcfg - 1.119ubuntu1

---------------
netcfg (1.119ubuntu1) utopic; urgency=medium

  * Resynchronise with Debian. Remaining changes:
    - Set default hostname to 'ubuntu'.
    - Set priority for get_domain to high for static configurations.
    - Set priority for get_domain to medium for non-static configurations.
    - Use 'auto <interface>' for all interfaces, dropping allow-hotplug
      which doesn't work with current udev.
    - Set DHCP and DHCPv6 timeout to 30s.
    - Use isc-dhcp-client-udeb on all architectures.
    - Flush all addresses and routes before configuring interfaces
      (LP: #848072)
    - Don't copy /etc/resolv.conf to target if resolvconf is installed. (We
      already write resolvconf configuration to /etc/network/interfaces.)
    - Add a post-base-install hook to detect resolvconf and copy
      /run/resolvconf/resolv.conf to outside the target so that when /run is
      bind-mounted DNS resolving continues to work. (LP: #926447)
    - Apply patch from Alec Warner making netcfg respect
      netcfg/dhcpv6_timeout and running dhclient in one-shot mode (-1).
      (LP: #917905)
    - Fix FTBFS by checking the return value of fgets and fscanf.
    - Fix nm-conf to generate a valid NetworkManager static configuration
      file.

netcfg (1.119) unstable; urgency=medium

  [ Colin Watson ]
  * Don't copy /etc/network/interfaces to /target if
    netcfg/target_network_config=ifupdown; it's already been copied by the
    base-installer hook earlier, and copying it again makes it hard to
    modify /target/etc/network/interfaces in a preseed file (closes:
    #709017, LP: #1361902).
  * Fix BOOTIF detection to handle the newline at the end of a line read
    from /proc/cmdline (LP: #1350302).

  [ Updated translations ]
  * Estonian (et.po) by Mattias Põldaru

netcfg (1.118) unstable; urgency=high

  * Fix support for entering an ESSID manually, it was previously
    getting ignored (Closes: #757478). Thanks to James Cowgill for the
    report and the patch!

netcfg (1.117) unstable; urgency=low

  * Add lintian overrides for untranslatable debconf templates. I like
    when packages are lintian-clean...

  [ Updated translations ]
  * Bosnian (bs.po) by Amila Valjevčić
 -- Colin Watson <email address hidden> Thu, 28 Aug 2014 12:42:06 -0700

Changed in netcfg (Ubuntu):
status: Fix Committed → Fix Released
Revision history for this message
Julian Edwards (julian-edwards) wrote :

Can someone please tell me if this bug is still an issue after the netcfg fix?

Changed in maas:
status: Triaged → Incomplete
Revision history for this message
Newell Jensen (newell-jensen) wrote : Re: [Bug 1350302] Re: Deploying node with di on armhf/keystone can't find BOOTIF

Julian,

I will verify this for you within the next day or two.

Newell

On Mon, Sep 1, 2014 at 7:35 PM, Julian Edwards <email address hidden>
wrote:

> Can someone please tell me if this bug is still an issue after the
> netcfg fix?
>
> ** Changed in: maas
> Status: Triaged => Incomplete
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1350302
>
> Title:
> Deploying node with di on armhf/keystone can't find BOOTIF
>
> Status in MAAS:
> Incomplete
> Status in “netcfg” package in Ubuntu:
> Fix Released
>
> Bug description:
> Dropping to di shell:
>
> /var/log # cat syslog | grep BOOTIF
> Jul 30 12:18:26 kernel: [ 0.000000] Kernel command line: nomodeset
> netcfg/choose_interface=auto text priority=critical auto url=
> http://10.228.64.209/MAAS/metadata/latest/by-id/node-b83608ea-1708-11e4-b7f0-f0921cb4cd2c/?op=get_preseed
> locale=en_US hostname=awaaw domain=maas log_host=10.228.64.209 log_port=514
> -- earlyprintk console=ttyS0,9600n8 BOOTIF=01-fc-15-b4-17-d4-49
> Jul 30 12:18:38 netcfg[1944]: INFO: Could not find valid BOOTIF= entry
> in /proc/cmdline
> Jul 30 12:19:35 netcfg[1944]: INFO: Could not find valid BOOTIF= entry
> in /proc/cmdline
> Jul 30 12:19:37 netcfg[2050]: INFO: Could not find valid BOOTIF= entry
> in /proc/cmdline
> Jul 30 12:23:03 netcfg[2050]: INFO: Could not find valid BOOTIF= entry
> in /proc/cmdline
> /var/log # ip link
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue
> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> 2: bond0: <BROADCAST,MULTICAST400> mtu 1500 qdisc noqueue
> link/ether 0a:09:aa:f8:b1:bf brd ff:ff:ff:ff:ff:ff
> 3: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc mq qlen 1000
> link/ether fc:15:b4:17:d4:48 brd ff:ff:ff:ff:ff:ff
> 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq qlen 1000
> link/ether fc:15:b4:17:d4:49 brd ff:ff:ff:ff:ff:ff
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/maas/+bug/1350302/+subscriptions
>

Revision history for this message
Colin Watson (cjwatson) wrote :

I've only uploaded this to utopic so far. Let me know if you need SRUs or whatever.

Revision history for this message
Narinder Gupta (narindergupta) wrote :

colin as HP is affected on this bug so this will be needed to SRUed for Trusty as well.

Revision history for this message
Colin Watson (cjwatson) wrote :

OK, that's fine. Please can you confirm first whether my fix for utopic works?

Changed in netcfg (Ubuntu Trusty):
status: New → Triaged
importance: Undecided → High
assignee: nobody → Colin Watson (cjwatson)
Revision history for this message
Newell Jensen (newell-jensen) wrote :

Colin,

I am doing a bunch of verification testing for MAAS in trusty so if it is
not too much of a pain getting this into trusty, that would be awesome. Or
if it makes it easier on you, maybe you can cut me a package that I can
test personally to verify this for you before you go through the entire SRU
process for it.

Thanks,

Newell

On Tue, Sep 9, 2014 at 2:20 PM, Colin Watson <email address hidden> wrote:

> OK, that's fine. Please can you confirm first whether my fix for utopic
> works?
>
> ** Also affects: netcfg (Ubuntu Trusty)
> Importance: Undecided
> Status: New
>
> ** Changed in: netcfg (Ubuntu Trusty)
> Status: New => Triaged
>
> ** Changed in: netcfg (Ubuntu Trusty)
> Importance: Undecided => High
>
> ** Changed in: netcfg (Ubuntu Trusty)
> Assignee: (unassigned) => Colin Watson (cjwatson)
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1350302
>
> Title:
> Deploying node with di on armhf/keystone can't find BOOTIF
>
> Status in MAAS:
> Incomplete
> Status in “netcfg” package in Ubuntu:
> Fix Released
> Status in “netcfg” source package in Trusty:
> Triaged
>
> Bug description:
> Dropping to di shell:
>
> /var/log # cat syslog | grep BOOTIF
> Jul 30 12:18:26 kernel: [ 0.000000] Kernel command line: nomodeset
> netcfg/choose_interface=auto text priority=critical auto url=
> http://10.228.64.209/MAAS/metadata/latest/by-id/node-b83608ea-1708-11e4-b7f0-f0921cb4cd2c/?op=get_preseed
> locale=en_US hostname=awaaw domain=maas log_host=10.228.64.209 log_port=514
> -- earlyprintk console=ttyS0,9600n8 BOOTIF=01-fc-15-b4-17-d4-49
> Jul 30 12:18:38 netcfg[1944]: INFO: Could not find valid BOOTIF= entry
> in /proc/cmdline
> Jul 30 12:19:35 netcfg[1944]: INFO: Could not find valid BOOTIF= entry
> in /proc/cmdline
> Jul 30 12:19:37 netcfg[2050]: INFO: Could not find valid BOOTIF= entry
> in /proc/cmdline
> Jul 30 12:23:03 netcfg[2050]: INFO: Could not find valid BOOTIF= entry
> in /proc/cmdline
> /var/log # ip link
> 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue
> link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
> 2: bond0: <BROADCAST,MULTICAST400> mtu 1500 qdisc noqueue
> link/ether 0a:09:aa:f8:b1:bf brd ff:ff:ff:ff:ff:ff
> 3: eth0: <BROADCAST,MULTICAST> mtu 1500 qdisc mq qlen 1000
> link/ether fc:15:b4:17:d4:48 brd ff:ff:ff:ff:ff:ff
> 4: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq qlen 1000
> link/ether fc:15:b4:17:d4:49 brd ff:ff:ff:ff:ff:ff
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/maas/+bug/1350302/+subscriptions
>

Revision history for this message
Ming Lei (tom-leiming) wrote :

Colin,

HP has marked it as critical in the private entry, and I have verified that netcfg_1.119 does fix the issue
on trusty, could you merge it to trusty ?

Thanks,

Revision history for this message
Raghuram Kota (rkota) wrote :

Hi Colin,

Thanks for working on the Trusty SRU for this :).

Just to elaborate on Ming's comment #12, this issue is currently blocking a Ubuntu factory image for HP's Moonshot platform. To unblock them, we'd first need this fix and then re-spin an installer, after which, HP needs some lead time to test and integrate the image into their factory process by next week.

Given the long chain of events, anything you can do to help expedite the availability of this fix in Trusty (preferably this week ) would be a great help!

Thanks much,
Raghu

Revision history for this message
dann frazier (dannf) wrote :

Here's the obvious backport for trusty.

dann frazier (dannf)
description: updated
Colin Watson (cjwatson)
Changed in netcfg (Ubuntu Trusty):
status: Triaged → In Progress
Revision history for this message
Chris J Arges (arges) wrote : Please test proposed package

Hello Newell, or anyone else affected,

Accepted netcfg into trusty-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/netcfg/1.116ubuntu2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in netcfg (Ubuntu Trusty):
status: In Progress → Fix Committed
tags: added: verification-needed
Revision history for this message
Raghuram Kota (rkota) wrote :

Colin,

HP successfully tested this fix along with the proposed installer from : http://ports.ubuntu.com/ubuntu-ports/dists/trusty-proposed/main/installer-armhf/20101020ubuntu318.7/images/

They reported success via comments #24, #25 &#26 of LP1370679.

If this sufficient to mark this "verification-done" can you please help do that ?

Thanks,
Raghu

Revision history for this message
Colin Watson (cjwatson) wrote :

I'm not sure why changing a bug tag needs my help, but sure, done :-)

tags: added: verification-done
removed: verification-needed
Revision history for this message
Raghuram Kota (rkota) wrote :

Thanks Colin. I just wanted to get your confirmation that testing done was sufficient prior to marking "verficiation-done". Thanks for confirming and adding the tag :-)

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package netcfg - 1.116ubuntu2

---------------
netcfg (1.116ubuntu2) trusty; urgency=medium

  * Fix BOOTIF detection to handle the newline at the end of a line read
    from /proc/cmdline (LP: #1350302).
 -- Colin Watson <email address hidden> Tue, 23 Sep 2014 20:38:54 +0100

Changed in netcfg (Ubuntu Trusty):
status: Fix Committed → Fix Released
Revision history for this message
Adam Conrad (adconrad) wrote : Update Released

The verification of the Stable Release Update for netcfg has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Newell Jensen (newell-jensen) wrote :

Julian,

The fix was tested with MAAS 1.5.4 as part of the certification for the Slayton and McDivitt platforms and is no longer an issue with regards to MAAS. The new package is now in trusty-updates as mentioned above.

Changed in maas:
status: Incomplete → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.