[SRU][xenial]boot stalls looking for entropy in FIPS mode

Bug #1748310 reported by Vineetha Kamath
12
This bug affects 1 person
Affects Status Importance Assigned to Milestone
libgcrypt20 (Ubuntu)
Fix Released
Undecided
Vineetha Kamath
Xenial
Fix Released
Undecided
Vineetha Kamath

Bug Description

[IMPACT]
libgcrypt20 is not a FIPS certified library. On a machine running FIPS enabled kernel, the library by default goes into FIPS mode if /proc/sys/crypto/fips_enabled=1. FIPS mode is not a configurable compile option currently in the library. Hence FIPS code paths are always executed on a FIPS enabled machine. In FIPS mode, it runs self tests and integrity checks and it looks for quality entropy from /dev/random. Additionally in desktop environments, gnome keyring daemon also queries libgcrypt for /dev/random entropy, slowing down the GUI startup.

On encrypted installations, cryptsetup uses libgcrypt20. During boot on an encrypted machine running in FIPS mode, cryptsetup invokes libgcrypt and it stalls looking for quality entropy from /dev/random. This results in significant delays during startup. The issue was reported by a FIPS customer.

The issue impacts libgcrypt versions in xenial and bionic.

lsb_release -rd
Description: Ubuntu 16.04.3 LTS
Release: 16.04

version - 1.6.5-2ubuntu0.3

lsb_release -rd
Description: Ubuntu Bionic Beaver (development branch)
Release: 18.04

version - 1.8.1-4

[FIX]
This fix proposes to disable libgcrypt reading /proc/sys/crypto/fips_enabled. We only want fips certified modules
reading this file and running in fips mode. libgcrypt is not one of our
fips certified modules, so should not be reading this along with our fips certified modules to determine whether to run in fips mode. The libgcrypt fips code in xenial is outdated and some algorithms are no longer allowed by recent FIPS 140-2 standards.

However, users do have the option to create a /etc/gcrypt/fips_enabled
file, manually, and force libgcrypt to run in fips mode. We propose to
leave this as is, so as to not regress anyone who is using this option.
We believe a user who uses this option is doing so with awareness.

[TEST]
Tested on a VM installed with xenial desktop iso and one with xenial server iso. Enabled full disk encryption during install. Tested with and without FIPS. No delays were observed during boot after the fix patch was applied.

Tested on a VM installed with Bionic development release version of desktop ISO with full disk encryption. Installed the xenial FIPS kernel and installed the fixed libgcrypt and did not observe any delays during the boot.

With FIPS enabled on encrypted install, without the patch fix, the boot stalls before and after prompting for decryption password. In desktop installations, a delay is observed during the GUI startup as well.

[REGRESSION POTENTIAL]
The regression potential for this is small. A fips kernel is required to
create /proc/sys/crypto/fips_enabled. For users forcing fips mode via
/etc/gcrypt/fips_enabled or the control option in libgcrypt, nothing has
changed.

tags: added: xenial
summary: - boot stalls looking for entropy in FIPS mode
+ [SRU][xenial]boot stalls looking for entropy in FIPS mode
Changed in libgcrypt20 (Ubuntu):
assignee: nobody → Vineetha Hari Pai (vineetha)
description: updated
description: updated
description: updated
Revision history for this message
Vineetha Kamath (vineetha) wrote :
Revision history for this message
Vineetha Kamath (vineetha) wrote :

debdiff.xenial

description: updated
description: updated
Revision history for this message
Vineetha Kamath (vineetha) wrote :

Please read comment #1 as build log and test run.

description: updated
Revision history for this message
Vineetha Kamath (vineetha) wrote :

Please ignore earlier comments. The fix was updated to remove self tests.

The build logs and tests run for xenial is here - https://launchpadlibrarian.net/357025470/buildlog_ubuntu-xenial-amd64.libgcrypt20_1.6.5-2ubuntu0.3+xenial.1_BUILDING.txt.gz

Revision history for this message
Vineetha Kamath (vineetha) wrote :

xenial debdiff

Revision history for this message
Vineetha Kamath (vineetha) wrote :
Revision history for this message
Vineetha Kamath (vineetha) wrote :

bionic debdiff

Revision history for this message
Vineetha Kamath (vineetha) wrote :

xenial package build is available on my ppa here - https://launchpad.net/~vineetha/+archive/ubuntu/gcrypt-xenial/

description: updated
description: updated
description: updated
Revision history for this message
Vineetha Kamath (vineetha) wrote :

We have zeroed in on a better solution of disabling reading fips_enabled file read on a FIPS system than the previous patches. Please ignore the diffs in previous comments.

The xenial build and test runs are here - https://launchpadlibrarian.net/357322446/buildlog_ubuntu-xenial-amd64.libgcrypt20_1.6.5-2ubuntu0.3+xenial.2_BUILDING.txt.gz

The xenial package is here - https://launchpad.net/~vineetha/+archive/ubuntu/gcrypt-xenial/+packages

The bionic build and tests runs are here - https://launchpadlibrarian.net/357327994/buildlog_ubuntu-bionic-amd64.libgcrypt20_1.8.1-4+bionic.2_BUILDING.txt.gz

The bionic package build is here - https://launchpad.net/~vineetha/+archive/ubuntu/libgcrypt-bionic/+packages

Revision history for this message
Vineetha Kamath (vineetha) wrote :
Revision history for this message
Vineetha Kamath (vineetha) wrote :
Revision history for this message
Marc Deslauriers (mdeslaur) wrote :

ACK on the debdiffs in comments #10 and #11. I've uploaded them to bionic and to xenial for processing by the SRU team with a slight change to the version number and LP tag.

Thanks!

Changed in libgcrypt20 (Ubuntu Xenial):
status: New → In Progress
Changed in libgcrypt20 (Ubuntu):
status: New → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package libgcrypt20 - 1.8.1-4ubuntu1

---------------
libgcrypt20 (1.8.1-4ubuntu1) bionic; urgency=medium

  * Disable the library reading /proc/sys/crypto/fips_enabled file
    and going into FIPS mode. libgcrypt is not a FIPS certified library.
    (LP: #1748310)
    - debian/patches/disable_fips_enabled_read.patch

 -- Vineetha Pai <email address hidden> Fri, 16 Feb 2018 13:45:04 -0500

Changed in libgcrypt20 (Ubuntu):
status: Fix Committed → Fix Released
Revision history for this message
Robie Basak (racb) wrote :

Hi Vineetha,

To help me understand the user impact, is /proc/sys/crypto/fips_enabled ever 1 on any kernel shipped by Ubuntu itself (so excluding the Canonical FIPS kernel)?

Revision history for this message
Vineetha Kamath (vineetha) wrote :

Hi Robie,

For any kernel shipped by Canonical (excluding the Canonical FIPS kernel), /proc/sys/crypto/fips_enabled file does not exist.

The kernel has to be compiled with "CONFIG_CRYPTO_FIPS" for the file to be even created and then based on the kernel command line parameters fips=1 or fips=0, the kernel sets value in that file. This is done only by a FIPS kernel.

Revision history for this message
Robie Basak (racb) wrote :

Thanks Vineetha.

To clarify for any observers, here's my understanding:

Ubuntu doesn't ship with a FIPS kernel by default.

If a user does use a FIPS enabled kernel, then libgcrypt20 detects this and activates its own FIPS mode.

libgcrypt20 in Xenial's FIPS mode requires using /dev/random, which can block when using full disk encryption during early boot up. This issue is tracked in this bug. The Xenial version of libcrypt code is outdated with respect to latest FIPS 140-2 specifications and parts of it uses non-compliant algorithms.

Our fix is to disable libgcrypt20's FIPS mode, which is fine because it isn't certified anyway.

This shouldn't regress existing users because Ubuntu's kernel does not ship with FIPS enabled and so this mode in libgcrypt20 doesn't activate in the usual case anyway.

For users who are using a FIPS-enabled kernel, Vineetha and her team will perform SRU verification as normal to test the update once it is in proposed.

We've tweaked the changelog entry for the SRU a little to make it clear that users who aren't using a FIPS kernel shouldn't be affected by this SRU.

Changed in libgcrypt20 (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-xenial
Revision history for this message
Robie Basak (racb) wrote : Please test proposed package

Hello Vineetha, or anyone else affected,

Accepted libgcrypt20 into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/libgcrypt20/1.6.5-2ubuntu0.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Revision history for this message
Vineetha Kamath (vineetha) wrote :

I tested libgcrypt20 (1.6.5-2ubuntu0.4) from xenial-proposed with the following configurations -
a) On a xenial VM running 16.04.3 server ISO with encrypted installation and fips enabled, the package fixes boot delays. Tested with both fips=1 and fips=0 and both cases work with no issues.

b) On a xenial VM running 16.04.3 desktop ISO with encrypted installation and fips enabled, the package fixes boot delays. Tested with both fips=1 and fips=0 and both cases work with no issues.

Revision history for this message
Vineetha Kamath (vineetha) wrote :

Details of the VM tested on.
 cat /etc/os-release
NAME="Ubuntu"
VERSION="16.04.3 LTS (Xenial Xerus)"
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME="Ubuntu 16.04.3 LTS"
VERSION_ID="16.04"
HOME_URL="http://www.ubuntu.com/"
SUPPORT_URL="http://help.ubuntu.com/"
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/"
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

guestvm@sru-libgcrypt-server:~$ dpkg -l | grep libgcrypt
ii libgcrypt20:amd64 1.6.5-2ubuntu0.4 amd64 LGPL Crypto library - runtime library

Changed in libgcrypt20 (Ubuntu Xenial):
assignee: nobody → Vineetha Hari Pai (vineetha)
tags: added: verification-done verification-done-xenial
removed: verification-needed verification-needed-xenial
Revision history for this message
Alex Stuart (astuart) wrote :

I have tested libgcrypt20_1.6.5-2ubuntu0.4 on roughly ten 16.04.3 desktop installations with encrypted root filesystems and the fips modules enabled. This patch appears to correct the bug.

Revision history for this message
Nivedita Singhvi (niveditasinghvi) wrote :

Thanks for the testing and update, Alex!

Revision history for this message
Łukasz Zemczak (sil2100) wrote :

The gvfs autopkgtest is also failing on vanilla gvfs - ignoring failure and releasing.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package libgcrypt20 - 1.6.5-2ubuntu0.4

---------------
libgcrypt20 (1.6.5-2ubuntu0.4) xenial; urgency=medium

  * Disable the library reading /proc/sys/crypto/fips_enabled file
    and going into FIPS mode. This fixes a hang on boot when using a
    FIPS-enabled kernel with encrypted installations (LP: #1748310)
    - debian/patches/disable_fips_enabled_read.patch

 -- Vineetha Pai <email address hidden> Fri, 16 Feb 2018 13:31:19 -0500

Changed in libgcrypt20 (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for libgcrypt20 has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Andreas Metzler (k-launchpad-downhill-at-eu-org) wrote :

Just for reference: "libgcrypt20 is not a FIPS certified library" was quite unclear to me. Both Red Hat and Suse have finished FIPS certifications for libgcrypt (for specific versions included in their respective enterprise distributions). Afaict Ubuntu has not run through this process at all, and therefore afaiui nothing on Ubuntu is "FIPS certified".

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.