MAC addresses hard-coded in bootloader

Bug #1900879 reported by James Vaughn
10
This bug affects 1 person
Affects Status Importance Assigned to Milestone
snap-core18
Fix Released
Critical
Unassigned

Bug Description

Posting this on behalf of a customer:

After a significant amount of support tickets over the years with customers complaining about colliding MAC addresses, we have *finally* been able to get our heads around *why* this is.

Backstory:
A common scenario for customers is that they have a single Raspberry Pi at their office at HQ (or similar) where they prepare their devices. They flash out the SD cards and once they're done setting the devices up, they either ship the SD cards out to the location (where they might already have drop-shipped the hardware from the supplier directly).

The reason why this is relevant is because Core will *hardcode* the MAC Address as part of the SD card (in /boot/uboot/uboot.env). Thus, when the device(s) are brought online at site, they will cause mayhem on the network, as the network gear assumes all hardware has a unique MAC Address.

The reason why we were finally able to reproduce this ourselves was because a similar workflow. We created 48 SD cards that we shipped off to a data center for our QC rig. When they booted up in their new home, the network collapsed because it had 48 devices all using the same MAC Address.

Reproducing this is easy:
* Flash out a Core18 image to an SD cards
* Boot it up on a Raspberry Pi and observe the MAC addresses
* Shut the device down
* Move it to a different Raspberry Pi and boot it up

You will now notice that the MAC Address is the same.

We used fw_printenv to find the values set in uboot.env and then remove them with fw_setenv. Unfortunately, this did not update uboot.env.

Tags: fr-851
Michael Vogt (mvo)
Changed in snap-core18:
status: New → Confirmed
importance: Undecided → Critical
Revision history for this message
Dave Jones (waveform) wrote :

This is occurring because the Core 18 boot environment uses "saveenv" to save its state, which dumps the *entire* U-Boot environment (including settings like ethaddr, and serial#) to disk. A workaround, as you've noted, is to edit the U-Boot environment in place to remove these settings. In the following GitHub gist you should find a small Python script I wrote a while ago (and just polished up a bit) which can accomplish this:

  https://gist.github.com/waveform80/62f3cc34dc87b8c26e6febc7f28c404e

Just save the script to a file named "edit-env", make it executable, and run it with the name of the U-Boot environment you wish to edit. It should open the environment as a series of key=value lines in the system's default editor. For example:

  $ wget https://gist.githubusercontent.com/waveform80/62f3cc34dc87b8c26e6febc7f28c404e/raw/9c18f5fc3e891307c874ed7e1b549ebad0828857/edit-env
  $ chmod +x edit-env
  $ ./edit-env uboot.env

You will need to delete the following settings from the environment:

  serial#
  ethaddr
  usbethaddr

serial# must be deleted as well as the ethaddr value is partially derived from it, and just to pre-empt the obvious question: yes, U-Boot sets the board's serial number in the device-tree from that value so all the boards in question will currently have the same serial number at runtime.

In the meantime, I'll see if I can whip up a more sensible core18 boot environment.

Revision history for this message
Viktor Petersson (vpetersson) wrote :

Just for clarity, this also affects Core16. Unclear if it is the same exact but tho.

Revision history for this message
Viktor Petersson (vpetersson) wrote :

Here are some more debug info on Core16:

https://gist.github.com/vpetersson/85399f22a4369789f1522e456c359a16

In the experiment, I moved the SD card from a 3 Model B to a 3 Model B+.

Revision history for this message
Viktor Petersson (vpetersson) wrote :

Update: I've confirmed that the fix does appear to work for Core16 too. More details in the gist.

Revision history for this message
Dave Jones (waveform) wrote :

I've just uploaded a fix for this to the 18-armhf and 18-arm64 branches of the pi-gadget repo (tested on 3B+ and 3B, moving card between machines and confirming boot still operates but MAC address, and serial#, both change). Here's the commit for 18-armhf which I've cherry-picked over to the 18-arm64 branch:

https://github.com/snapcore/pi-gadget/commit/d837f0cfcdc84b92b3de8a380bafd14e55e65f6a

This won't affect core20 which uses a rather different mechanism to core18 for boot state storage (doesn't re-write the entire environment, just stores a minimal separate state). I'll look into what we can do for core16 (I've just checked the image and my assumption from looking at the boot environment is that it's very likely the exact same thing).

tags: added: fr-851
Dave Jones (waveform)
Changed in snap-core18:
status: Confirmed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.