/usr/share/apport/whoopsie-upload-all:zlib.error:/usr/share/apport/whoopsie-upload-all@196:collect_info:process_report:add_gdb_info:gdb_command:write:read:readinto:read

Bug #1947800 reported by errors.ubuntu.com bug bridge
52
This bug affects 2 people
Affects Status Importance Assigned to Milestone
Apport
Fix Released
High
Unassigned
apport (Ubuntu)
Fix Released
High
Unassigned
Focal
Fix Released
High
Unassigned
Impish
Won't Fix
High
Unassigned
Jammy
Fix Released
High
Unassigned

Bug Description

[ Impact ]

The Ubuntu Error Tracker has been receiving reports about a problem regarding apport. This problem was most recently seen with package version 2.20.11-0ubuntu70, the problem page at https://errors.ubuntu.com/problem/7120f8aebdf09e9dca39196d035eec234071e800 contains more details, including versions of packages affected, stacktrace or traceback, and individual crash reports.
If you do not have access to the Ubuntu Error Tracker and are a software developer, you can request it at http://forms.canonical.com/reports/.

Traceback (most recent call last):
  File "/usr/share/apport/whoopsie-upload-all", line 196, in <module>
    stamps = collect_info()
  File "/usr/share/apport/whoopsie-upload-all", line 146, in collect_info
    res = process_report(r)
  File "/usr/share/apport/whoopsie-upload-all", line 103, in process_report
    r.add_gdb_info()
  File "/usr/lib/python3/dist-packages/apport/report.py", line 786, in add_gdb_info
    gdb_cmd, environ = self.gdb_command(rootdir, gdb_sandbox)
  File "/usr/lib/python3/dist-packages/apport/report.py", line 1706, in gdb_command
    self['CoreDump'].write(f)
  File "/usr/lib/python3/dist-packages/problem_report.py", line 75, in write
    block = gz.read(1048576)
  File "/usr/lib/python3.9/gzip.py", line 300, in read
    return self._buffer.read(size)
  File "/usr/lib/python3.9/_compression.py", line 68, in readinto
    data = self.read(len(byte_view))
  File "/usr/lib/python3.9/gzip.py", line 495, in read
    uncompress = self._decompressor.decompress(buf, size)
zlib.error: Error -3 while decompressing data: invalid code lengths set

Report for 20.04 (focal): https://errors.ubuntu.com/problem/fff3109d511a7db05ec5526679b8d8608a143932

[ Test plan ]

Check that the fixed version does not appear on
https://errors.ubuntu.com/problem/fff3109d511a7db05ec5526679b8d8608a143932

[Where problems could occur]

Worst case - in any part of the bug reporting/collection procedure, since that is what's changing.

tags: added: rls-ii-incoming
tags: added: fr-1816
tags: removed: rls-ii-incoming
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in apport (Ubuntu Impish):
status: New → Confirmed
Changed in apport (Ubuntu):
status: New → Confirmed
Changed in apport (Ubuntu Jammy):
importance: Undecided → High
Changed in apport (Ubuntu Impish):
importance: Undecided → High
Benjamin Drung (bdrung)
description: updated
Benjamin Drung (bdrung)
tags: added: jammy
Revision history for this message
Benjamin Drung (bdrung) wrote (last edit ):

Bug analysis
============

whoopsie-upload-all reads the *.crash files (to upload them):

```
r = apport.Report()
with open(report, 'rb') as f:
    r.load(f, binary='compressed')
```

ProblemReport.load reads the encoded data and keeps in compressed:

```python
block = base64.b64decode(line)
if value.gzipvalue == b'' and not block.startswith(b'\037\213\010'):
    value.legacy_zlib = True
value.gzipvalue += block
```

Derived from the code path, `legacy_zlib` is set to `False` in this case. Then `Report.gdb_command` tries to decompress the CoreDump field and write it into a temporary file. The relevant part from `problem_report.py` from apport 2.20.11-0ubuntu71 (line 73):

```python
gz = gzip.GzipFile(fileobj=BytesIO(self.gzipvalue))
while True:
    block = gz.read(1048576)
    if not block:
        break
    file.write(block)
```

So the only two reasons that I can see why decompressing the byte object `gzipvalue`:
1. The content of the CoreDump field in the .crash file is malformed.
2. `legacy_zlib` is set to `False` instead of `True`.

Reproduction
============

I tried to reproduce the `zlib.error` by feeding it with wrong data:

```
In [1]: import gzip

In [2]: from io import BytesIO

In [3]: gzip.GzipFile(fileobj=BytesIO(None)).read(1048576)
Out[3]: b''

In [4]: gzip.GzipFile(fileobj=BytesIO(b"")).read(1048576)
Out[4]: b''

In [5]: gzip.GzipFile(fileobj=BytesIO(b"bogus")).read(1048576)
[...]
BadGzipFile: Not a gzipped file (b'bo')
```

Revision history for this message
Benjamin Drung (bdrung) wrote :

Let's catch the zlib.error exception the same way as the other exceptions are caught.

tags: added: patch
Revision history for this message
Benjamin Drung (bdrung) wrote :

Reproduction Part 2
===================

I used divide-by-zero from chaos-marmosets to create a crash file and deleted parts of CoreDump. Then I used following snippet to test the modified crash file:

```python
import sys, apport

report = sys.argv[1]
r = apport.Report()
with open(report, 'rb') as f:
    r.load(f, binary='compressed')
r.add_gdb_info()
```

I only got these three types of call traces:

```
EOFError: Compressed file ended before the end-of-stream marker was reached
binascii.Error: Incorrect padding
binascii.Error: Invalid base64-encoded string: number of data characters (21189) cannot be 1 more than a multiple of 4
```

Chaning one character in CoreDump triggered a different error message:

```
gzip.BadGzipFile: CRC check failed 0xfb18ba58 != 0x7f1b4388
```

So it is unclear to me how to produce a faulty CoreDump entry in the crash report.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apport - 2.20.11-0ubuntu82

---------------
apport (2.20.11-0ubuntu82) jammy; urgency=medium

  * whoopsie-upload-all:
    - Catch zlib.error when decoding CoreDump from crash file (LP: #1947800)
    - Catch FileNotFoundError during process_report (LP: #1867204)

 -- Benjamin Drung <email address hidden> Wed, 13 Apr 2022 22:20:41 +0200

Changed in apport (Ubuntu Jammy):
status: Confirmed → Fix Released
Benjamin Drung (bdrung)
description: updated
description: updated
Benjamin Drung (bdrung)
Changed in apport:
status: New → Fix Released
importance: Undecided → Critical
importance: Critical → High
milestone: none → 2.21.0
Revision history for this message
Benjamin Drung (bdrung) wrote :
Revision history for this message
Benjamin Drung (bdrung) wrote :

Since Ubuntu 21.10 (impish) will be end of life soon, we won't prepare a SRU for it.

Changed in apport (Ubuntu Impish):
status: Confirmed → Won't Fix
Changed in apport (Ubuntu Focal):
importance: Undecided → High
Benjamin Drung (bdrung)
Changed in apport (Ubuntu Focal):
status: New → Fix Committed
Benjamin Drung (bdrung)
Changed in apport (Ubuntu Focal):
status: Fix Committed → New
Benjamin Drung (bdrung)
description: updated
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in apport (Ubuntu Focal):
status: New → Confirmed
Revision history for this message
Steve Langasek (vorlon) wrote : Please test proposed package

Hello errors.ubuntu.com, or anyone else affected,

Accepted apport into focal-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.11-0ubuntu27.27 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, what testing has been performed on the package and change the tag from verification-needed-focal to verification-done-focal. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-focal. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in apport (Ubuntu Focal):
status: Confirmed → Fix Committed
tags: added: verification-needed verification-needed-focal
Revision history for this message
Benjamin Drung (bdrung) wrote :
tags: added: verification-done verification-done-focal
removed: verification-needed verification-needed-focal
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apport - 2.20.11-0ubuntu27.27

---------------
apport (2.20.11-0ubuntu27.27) focal; urgency=medium

  * whoopsie-upload-all: Catch zlib.error when decoding CoreDump from
    crash file (LP: #1947800)
  * Fix KeyError: 'CasperMD5json' (LP: #1964828)
  * apport-kde: Fix inverse order of choices (LP: #1967965)
  * apport-unpack: Fix ValueError: ['separator'] has no binary content
    (LP: #1889443)
  * test:
    - Clear environment for test_run_as_real_user_no_sudo
    - Mock add_gdb_info calls in KDE UI tests
    - Fix KDE UI tests if whoopsie.path is disabled
    - Fix race with progress dialog in KDE UI tests
    - Run UI KDE tests again
    - Determine source package dynamically in test_run_crash_kernel
      (LP: #1992172)

 -- Benjamin Drung <email address hidden> Fri, 14 Apr 2023 01:17:21 +0200

Changed in apport (Ubuntu Focal):
status: Fix Committed → Fix Released
Revision history for this message
Chris Halse Rogers (raof) wrote : Update Released

The verification of the Stable Release Update for apport has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.