Ubuntu
python2.7 package

FIPS OpenSSL crashes Python2 hashlib

Bug #1835135 reported by William Ahern
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
python2.7 (Ubuntu)
Triaged
High
Unassigned
Xenial
Fix Released
Medium
Marc Deslauriers
Bionic
Fix Released
Medium
Marc Deslauriers
Cosmic
Won't Fix
Undecided
Unassigned
Disco
Fix Released
Medium
Marc Deslauriers
Eoan
Won't Fix
High
Unassigned
python3.5 (Ubuntu)
Invalid
Undecided
Unassigned
Xenial
Fix Released
Medium
Marc Deslauriers
Bionic
Invalid
Undecided
Unassigned
Cosmic
Invalid
Undecided
Unassigned
Disco
Invalid
Undecided
Unassigned
Eoan
Invalid
Undecided
Unassigned

Bug Description

If Ubuntu/Canonical's FIPS-compliant OpenSSL is initialized with SSL_library_init, then Python2's hashlib bindings for MD5 can trigger a SIGSEGV via a NULL pointer dereference (if calling the .update method) or a SIGABRT (if passing input to the constructor or passing no input and invoking the .final method). This happens if, for example, PyOpenSSL is imported before hashlib.

Canonical's FIPS patches for OpenSSL introduce some odd behavior that arguably should be revisited, but the (TL;DR) core bug is that Python2 hashlib doesn't properly check the return value of EVP_DigestInit, preventing hashlib from falling back to it's internal MD5 implementation and instead setting things up for use of the MD5 context to trigger SIGSEGV or SIGABRT.

Python3 correctly checks the return value, so the fix is to backport the relevant code into Python2 (see python2.7-2.7.12/Modules/_hashopenssl.c).

See attached good.py and bad.py files which exhibit the import order-dependent crashing issue. See attached fips-md5-python-init-bug.c which shows the FIPS OpenSSL behaviors that conditionally tickle the Python2 bug. The C file also contains a much more detailed description of the Python2 bug and other behavior which I'd rather not repeat here.

I discovered this bug investigating an issue with the third-party apt-boto-s3 package. See https://github.com/boto/boto3/issues/2021

Note that this bug effects Splunk, Inc, which has a corporate Ubuntu Advantage license. My login account is attached to a different, single-seat license.

Revision history for this message
William Ahern (wahern-splunk) wrote :
Revision history for this message
Andreas Hasenack (ahasenack) wrote :

Thanks for bringing this up. The FIPS team is aware of it and will address this.

Changed in python2.7 (Ubuntu):
status: New → Triaged
importance: Undecided → High
Revision history for this message
Joy Latten (j-latten) wrote :

Investigating

Revision history for this message
Joy Latten (j-latten) wrote :

The assessment is accurate.

FIPS 140-2 does not allow MD5 except for use in PRF.

Thus the OpenSSL_add_all_digests in fips openssl does not include MD5. However, SSL_library_init() does include MD5 but only for use in calculating the PRF. Notice in tls1_P_hash() in ssl/t1_enc.c
the flag, EVP_MD_CTX_FLAG_NON_FIPS_ALLOW, is set in the context to permit this use of MD5.
Apps wishing to calculate their own PRF can do the same.

Revision history for this message
Joy Latten (j-latten) wrote :

Like python3, python2 should check the return value of EVP_DigestInit.

Revision history for this message
Joy Latten (j-latten) wrote :

Upon looking at the source for both python2.7 and python3.5 in xenial, neither checks the return value from EVP_DigestInit in Modules/_hashopenssl.c file.

However, python3.6 (in bionic, cosmic and disco) does have the check.

So the check will need to be backported to python 2.7 and python 3.5 in xenial.

Marc Deslauriers (mdeslaur)
Changed in python3.5 (Ubuntu Bionic):
status: New → Invalid
Changed in python3.5 (Ubuntu Cosmic):
status: New → Invalid
Changed in python3.5 (Ubuntu Disco):
status: New → Invalid
Changed in python3.5 (Ubuntu Eoan):
status: New → Invalid
Changed in python3.5 (Ubuntu Xenial):
assignee: nobody → Marc Deslauriers (mdeslaur)
importance: Undecided → Medium
status: New → In Progress
Changed in python2.7 (Ubuntu Xenial):
assignee: nobody → Marc Deslauriers (mdeslaur)
importance: Undecided → Medium
status: New → In Progress
Changed in python2.7 (Ubuntu Bionic):
assignee: nobody → Marc Deslauriers (mdeslaur)
importance: Undecided → Medium
status: New → In Progress
Changed in python2.7 (Ubuntu Cosmic):
status: New → Won't Fix
Changed in python2.7 (Ubuntu Disco):
assignee: nobody → Marc Deslauriers (mdeslaur)
importance: Undecided → Medium
status: New → In Progress
Revision history for this message
Joy Latten (j-latten) wrote :

The 2.7 and 3.5 python packages in the security proposed PPA have been successfully tested in a fips and non-fips xenial environment.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python2.7 - 2.7.15-4ubuntu4~18.04.1

---------------
python2.7 (2.7.15-4ubuntu4~18.04.1) bionic-security; urgency=medium

  * SECURITY UPDATE: incorrect cookie domain check
    - debian/patches/CVE-2018-20852.patch: prefix dot in domain for proper
      subdomain validation in Lib/cookielib.py, Lib/test/test_cookielib.py.
    - CVE-2018-20852
  * SECURITY UPDATE: NULL pointer dereference via X509 certificate
    - debian/patches/CVE-2019-5010.patch: fix segfault in ssl cert parser
      in Lib/test/talos-2019-0758.pem, Lib/test/test_ssl.py,
      Modules/_ssl.c.
    - CVE-2019-5010
  * SECURITY UPDATE: improper handling of unicode encoding
    - debian/patches/CVE-2019-9636-1.patch: add check for characters in
      netloc that normalize to separators in Doc/library/urlparse.rst,
      Lib/test/test_urlparse.py, Lib/urlparse.py.
    - debian/patches/CVE-2019-9636-2.patch: only print test messages when
      verbose in Lib/test/test_urlparse.py.
    - CVE-2019-9636
  * SECURITY UPDATE: HTTP header injection
    - debian/patches/CVE-2019-9740.patch: disallow control chars in http
      URLs in Lib/httplib.py, Lib/test/test_urllib.py,
      Lib/test/test_urllib2.py, Lib/test/test_xmlrpc.py.
    - CVE-2019-9740
    - CVE-2019-9947
  * SECURITY UPDATE: urllib support the local_file: scheme
    - debian/patches/CVE-2019-9948.patch: disallow file reading in
      Lib/urllib.py, Lib/test/test_urllib.py.
    - CVE-2019-9948
  * SECURITY UPDATE: incomplete fix for CVE-2019-9636
    - debian/patches/CVE-2019-10160-1.patch: fix handling of
      pre-normalization characters in urlsplit() in
      Lib/test/test_urlparse.py, Lib/urlparse.py.
    - debian/patches/CVE-2019-10160-2.patch: correct fix to handle
      decomposition in usernames in Lib/test/test_urlparse.py,
      Lib/urlparse.py.
    - debian/patches/CVE-2019-10160-3.patch: fix urlparse.urlsplit() error
      message for Unicode URL in Lib/test/test_urlparse.py,
      Lib/urlparse.py.
    - CVE-2019-10160
  * debian/patches/issue9146.diff: fix FIPS mode environments where MD5
    isn't available in Modules/_hashopenssl.c. (LP: #1835135)

 -- Marc Deslauriers <email address hidden> Tue, 09 Jul 2019 12:51:35 -0400

Changed in python2.7 (Ubuntu Bionic):
status: In Progress → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python2.7 - 2.7.16-2ubuntu0.1

---------------
python2.7 (2.7.16-2ubuntu0.1) disco-security; urgency=medium

  * SECURITY UPDATE: incorrect cookie domain check
    - debian/patches/CVE-2018-20852.patch: prefix dot in domain for proper
      subdomain validation in Lib/cookielib.py, Lib/test/test_cookielib.py.
    - CVE-2018-20852
  * SECURITY UPDATE: HTTP header injection
    - debian/patches/CVE-2019-9740.patch: disallow control chars in http
      URLs in Lib/httplib.py, Lib/test/test_urllib.py,
      Lib/test/test_urllib2.py, Lib/test/test_xmlrpc.py.
    - CVE-2019-9740
    - CVE-2019-9947
  * SECURITY UPDATE: incomplete fix for CVE-2019-9636
    - debian/patches/CVE-2019-10160-1.patch: fix handling of
      pre-normalization characters in urlsplit() in
      Lib/test/test_urlparse.py, Lib/urlparse.py.
    - debian/patches/CVE-2019-10160-2.patch: correct fix to handle
      decomposition in usernames in Lib/test/test_urlparse.py,
      Lib/urlparse.py.
    - debian/patches/CVE-2019-10160-3.patch: fix urlparse.urlsplit() error
      message for Unicode URL in Lib/test/test_urlparse.py,
      Lib/urlparse.py.
    - CVE-2019-10160
  * debian/patches/issue36216-2.diff: only print test messages when verbose
    in Lib/test/test_urlparse.py.
  * debian/patches/issue9146.diff: fix FIPS mode environments where MD5
    isn't available in Modules/_hashopenssl.c. (LP: #1835135)

 -- Marc Deslauriers <email address hidden> Tue, 09 Jul 2019 12:43:02 -0400

Changed in python2.7 (Ubuntu Disco):
status: In Progress → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python3.5 - 3.5.2-2ubuntu0~16.04.8

---------------
python3.5 (3.5.2-2ubuntu0~16.04.8) xenial-security; urgency=medium

  * SECURITY UPDATE: incorrect cookie domain check
    - debian/patches/CVE-2018-20852.patch: prefix dot in domain for proper
      subdomain validation in Lib/http/cookiejar.py,
      Lib/test/test_http_cookiejar.py.
    - CVE-2018-20852
  * SECURITY UPDATE: integer overflow in pickle
    - debian/patches/CVE-2018-20406.patch: avoid relying on signed overflow
      in _pickle memos in Modules/_pickle.c.
    - CVE-2018-20406
  * SECURITY UPDATE: NULL pointer dereference via X509 certificate
    - debian/patches/CVE-2019-5010.patch: fix segfault in ssl cert parser
      in Lib/test/talos-2019-0758.pem, Lib/test/test_ssl.py,
      Modules/_ssl.c.
    - CVE-2019-5010
  * SECURITY UPDATE: improper handling of unicode encoding
    - debian/patches/CVE-2019-9636.patch: add check for characters in
      netloc that normalize to separators in Doc/library/urllib.parse.rst,
      Lib/test/test_urlparse.py, Lib/urllib/parse.py.
    - CVE-2019-9636
  * SECURITY UPDATE: HTTP header injection
    - debian/patches/CVE-2019-9740.patch: disallow control chars in http
      URLs in Lib/http/client.py, Lib/test/test_urllib.py,
      Lib/test/test_xmlrpc.py.
    - CVE-2019-9740
    - CVE-2019-9947
  * SECURITY UPDATE: urllib support the local_file: scheme
    - debian/patches/CVE-2019-9948.patch: disallow file reading in
      Lib/urllib/request.py, Lib/test/test_urllib.py.
    - CVE-2019-9948
  * SECURITY UPDATE: incomplete fix for CVE-2019-9636
    - debian/patches/CVE-2019-10160-1.patch: fix handling of
      pre-normalization characters in urlsplit() in
      Lib/test/test_urlparse.py, Lib/urllib/parse.py.
    - debian/patches/CVE-2019-10160-2.patch: correct fix to handle
      decomposition in usernames in Lib/test/test_urlparse.py,
      Lib/urllib/parse.py.
    - CVE-2019-10160
  * debian/patches/issue9146.diff: fix FIPS mode environments where MD5
    isn't available in Modules/_hashopenssl.c. (LP: #1835135)

 -- Marc Deslauriers <email address hidden> Wed, 10 Jul 2019 07:58:48 -0400

Changed in python3.5 (Ubuntu Xenial):
status: In Progress → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package python2.7 - 2.7.12-1ubuntu0~16.04.8

---------------
python2.7 (2.7.12-1ubuntu0~16.04.8) xenial-security; urgency=medium

  * SECURITY UPDATE: incorrect cookie domain check
    - debian/patches/CVE-2018-20852.patch: prefix dot in domain for proper
      subdomain validation in Lib/cookielib.py, Lib/test/test_cookielib.py.
    - CVE-2018-20852
  * SECURITY UPDATE: NULL pointer dereference via X509 certificate
    - debian/patches/CVE-2019-5010.patch: fix segfault in ssl cert parser
      in Lib/test/talos-2019-0758.pem, Lib/test/test_ssl.py,
      Modules/_ssl.c.
    - CVE-2019-5010
  * SECURITY UPDATE: improper handling of unicode encoding
    - debian/patches/CVE-2019-9636-1.patch: add check for characters in
      netloc that normalize to separators in Doc/library/urlparse.rst,
      Lib/test/test_urlparse.py, Lib/urlparse.py.
    - debian/patches/CVE-2019-9636-2.patch: only print test messages when
      verbose in Lib/test/test_urlparse.py.
    - CVE-2019-9636
  * SECURITY UPDATE: HTTP header injection
    - debian/patches/bpo30500.patch: simplify splithost by calling into
      urlparse in Lib/test/test_urllib.py, Lib/urllib.py.
    - debian/patches/CVE-2019-9740.patch: disallow control chars in http
      URLs in Lib/httplib.py, Lib/test/test_urllib.py,
      Lib/test/test_urllib2.py, Lib/test/test_xmlrpc.py.
    - CVE-2019-9740
    - CVE-2019-9947
  * SECURITY UPDATE: urllib support the local_file: scheme
    - debian/patches/CVE-2019-9948.patch: disallow file reading in
      Lib/urllib.py, Lib/test/test_urllib.py.
    - CVE-2019-9948
  * SECURITY UPDATE: incomplete fix for CVE-2019-9636
    - debian/patches/CVE-2019-10160-1.patch: fix handling of
      pre-normalization characters in urlsplit() in
      Lib/test/test_urlparse.py, Lib/urlparse.py.
    - debian/patches/CVE-2019-10160-2.patch: correct fix to handle
      decomposition in usernames in Lib/test/test_urlparse.py,
      Lib/urlparse.py.
    - debian/patches/CVE-2019-10160-3.patch: fix urlparse.urlsplit() error
      message for Unicode URL in Lib/test/test_urlparse.py,
      Lib/urlparse.py.
    - CVE-2019-10160
  * debian/patches/issue9146.diff: fix FIPS mode environments where MD5
    isn't available in Modules/_hashopenssl.c. (LP: #1835135)

 -- Marc Deslauriers <email address hidden> Thu, 22 Aug 2019 12:36:40 -0400

Changed in python2.7 (Ubuntu Xenial):
status: In Progress → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote :

eoan has reached end of life, so this bug will not be fixed for that release

Changed in python2.7 (Ubuntu Eoan):
status: Triaged → Won't Fix
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.