dmidecode triggers system reboot on Inforce 6640

Thanks for filing a bug. I've added a dmidecode task to track the issue with the tool. It may also affect the kernel package, and possibly firmware (though that's not something that Ubuntu provides). Cloud-init and any other tool may invoke this package and it should not reboot the system; but there's no issue with cloud-init or other callers of the tool As such, I'm marking the cloud-init task as invalid.

@Ryan
Not all aarch64 platforms can support demidecode well.
How about revert the patch? or create a black list?

Ethan, cloud-init isn't the right level to handle hardware issues with DMI (which is why we've added the dmidecode task). I've pinged kernel folks to see if we can get eyes on this issue.

Does the kernel expose the DMI tables via the sysfs following sysfs file: /sys/firmware/dmi/tables/DMI ?

If so, can you do the following:

sudo cat /sys/firmware/dmi/tables/DMI > dmi.raw

and attach it to the bug report. Also a dump of the kernel dmesg log after it boots may be useful to see if it's a broken firmware DMI table or a kernel issue.

Oh, stupid me, I've just read the info in comment #1

@Dan
Got it. Thanks~

Hi, can you provide me instructions on how to get and install the image for this board? I'd like to reproduce this issue and get a suitable fix for this.

So, dmidecode directly mmap's to /dev/mem and does some probing based on the belief that the system is a x86 architecture even on arm architectures.

openat(AT_FDCWD, "/dev/mem", O_RDONLY) = 3
fstat(3, {st_mode=S_IFCHR|0640, st_rdev=makedev(0x1, 0x1), ...}) = 0
mmap(NULL, 65536, PROT_READ, MAP_SHARED, 3, 0xf0000) = 0x7f9f6fd000

etc

So that's kind of intrusive and as root one can read any sort of physical addresses in /dev/mem that may cause breakage.

dmidocode.c directly accesses memory and assumes it's an x86 without any checking that the arch is x86.. Randomly scanning arbitrary hunks of memory on non-x86 as root will lead to all sorts of woe:

memory_scan:
        if (!(opt.flags & FLAG_QUIET))
                printf("Scanning %s for entry point.\n", opt.devmem);
        /* Fallback to memory scan (x86, x86_64) */
        if ((buf = mem_chunk(0xF0000, 0x10000, opt.devmem)) == NULL)
        {
                ret = 1;
                goto exit_free;
        }

It probably needs wrapping with:

#if defined(__x86_64__) || defined(__x86_64) || \
    defined(__i386__) || defined(__i386)

...

#endif

Anyhow, I don't think this is a kernel specific issue. I can trigger this with various kernels - we just don't protect users with CAP_SYS_ADMIN rights doing crazy probing on /dev/mem.

I guess the next question is why dmidecode being run as root is required on a cloud init? What happens when arches don't have DMI data?

Upstream has a fix like the one I was hinting at in comment #9, I'll SRU this fix.

commit e12ec26e19e02281d3e7258c3aabb88a5cf5ec1d
Author: Jean Delvare <email address hidden>
Date: Mon Aug 26 14:20:15 2019 +0200

    dmidecode: Only scan /dev/mem for entry point on x86

    x86 is the only architecture which can have a DMI entry point scanned
    from /dev/mem. Do not attempt it on other architectures, because not
    only it can't work, but it can even cause the system to reboot.

    This fixes support request #109697:
    https://savannah.nongnu.org/support/?109697

We carry this patch in focal - do we just need to backport it to bionic?

http://git.savannah.nongnu.org/cgit/dmidecode.git/commit/?id=e12ec26e19e02281d3e7258c3aabb88a5cf5ec1d

I uploaded a test fix to ppa:dannf/test. Could someone w/ hw access verify that?

It needs backporting to eoan, disco bionic, I was just about to upload a fix to my ppa so I could get it sponsored. Do you want to take it from here Dann?

Sure Colin, I'll take it from here - thanks for your analysis so far. As a next step, I'll wait for Ethan - or someone else w/ hw access - to verify the PPA build in Comment #12.

Actually, I realized I can reproduce this on a dragonboard I have here. I've verified the PPA fix myself, so I'll go ahead and SRU.

On Mon, Jan 27, 2020 at 05:07:06PM -0000, Colin Ian King wrote:
> I guess the next question is why dmidecode being run as root is required
> on a cloud init?

cloud-init uses DMI data to identify that it is running on certain
platforms. It will use /sys/class/dmi/... if available, but falls back
to dmidecode in its absence.

> What happens when arches don't have DMI data?

If running on a platform which identifies itself only via DMI data,
cloud-init will incorrectly determine that it is _not_ running on that
platform.

Given that other arches _can_ provide DMI data, and this cloud-init
behaviour is known, I would be surprised if any platform that _relies_
on it would launch a new architecture without DMI support. So if we get
no data (instead of a reboot), then I believe the current behaviour is
correct.

Dann, tested on my 6640 on an older kernel, now get:

sudo dmidecode
# dmidecode 3.1
# No SMBIOS nor DMI entry point found, sorry.

I guess that's expected.

I'd like to see what Ethan gets on his H/W as I'm not running a cloud installation on my dev board.

Hello ethan.hsieh, or anyone else affected,

Accepted dmidecode into eoan-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/dmidecode/3.2-2ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-eoan to verification-done-eoan. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-eoan. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello ethan.hsieh, or anyone else affected,

Accepted dmidecode into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/dmidecode/3.1-1ubuntu0.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Hello ethan.hsieh, or anyone else affected,

Accepted dmidecode into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/dmidecode/3.0-2ubuntu0.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

All autopkgtests for the newly accepted dmidecode (3.2-2ubuntu0.1) for eoan have finished running.
The following regressions have been reported in tests triggered by the package:

crmsh/unknown (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/eoan/update_excuses.html#dmidecode

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

All autopkgtests for the newly accepted dmidecode (3.1-1ubuntu0.1) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

crmsh/3.0.1-3ubuntu1 (armhf)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#dmidecode

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

e/b/x all verified on a dragonboard 410c

Both crmsh autopkgtest failures could be explained by a flaky network connection. The eoan one is certainly that - it's unable to connect to the archive. The error from the bionic case is masked - but the test is trying to apt install vim. That could obviously also be an archive connectivity issue. I retried both tests, and they now appear to have passed.

@Colin @dann @Timo

I have verified the following packages on my inforce 6640.
eoan-proposed: dmidecode_3.2-2ubuntu0.1_arm64.deb
bionic-proposed: dmidecode_3.1-1ubuntu0.1_arm64.deb
xenial-proposed: dmidecode_3.0-2ubuntu0.2_arm64.deb

They look good.
When I run dmidecode, I get the following message instead of system reboot.

# dmidecode 3.X
# No SMBIOS nor DMI entry point found, sorry.

Verified dmidecode_(3.1-1ubuntu0.1_amd64.deb) on my x86 laptop (Ubuntu 18.04.3 LTS, Desktop).
It works well.

This bug was fixed in the package dmidecode - 3.2-2ubuntu0.1

---------------
dmidecode (3.2-2ubuntu0.1) eoan; urgency=medium

  * Restrict /dev/mem scanning for entry point to x86, fixing crashes
    on certain ARM platforms (LP: #1858615):
    - New debian/patches/0140-Fix_scan_entry_point.patch (Cherry picked from
      upstream Commit e12ec26e19e02281d3e7258c3aabb88a5cf5ec1d.

 -- dann frazier <email address hidden> Mon, 27 Jan 2020 10:55:41 -0700

The verification of the Stable Release Update for dmidecode has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

This bug was fixed in the package dmidecode - 3.1-1ubuntu0.1

---------------
dmidecode (3.1-1ubuntu0.1) bionic; urgency=medium

  * Restrict /dev/mem scanning for entry point to x86, fixing crashes
    on certain ARM platforms (LP: #1858615):
    - New debian/patches/0140-Fix_scan_entry_point.patch (Cherry picked from
      upstream Commit e12ec26e19e02281d3e7258c3aabb88a5cf5ec1d.

 -- dann frazier <email address hidden> Mon, 27 Jan 2020 11:09:10 -0700

This bug was fixed in the package dmidecode - 3.0-2ubuntu0.2

---------------
dmidecode (3.0-2ubuntu0.2) xenial; urgency=medium

  * Restrict /dev/mem scanning for entry point to x86, fixing crashes
    on certain ARM platforms (LP: #1858615):
    - New debian/patches/0140-Fix_scan_entry_point.patch (Cherry picked from
      upstream Commit e12ec26e19e02281d3e7258c3aabb88a5cf5ec1d.

 -- dann frazier <email address hidden> Mon, 27 Jan 2020 11:29:42 -0700

Tracked in Github Issues as https://github.com/canonical/cloud-init/issues/3525