apport fails to collect nvram specific information -- attach_root_command_outputs() does not work for binary data

Please ignore the first diff. The second diff is the correct one. Thanks.

As fix is available changed status in Launchpad as I don't know how to do it from bugzilla

 diff -urN powerpc.py.old powerpc.py
--- powerpc.py.old 2014-09-04 06:33:16.000000000 -0400
+++ powerpc.py 2014-09-10 10:28:18.824002770 -0400
@@ -69,8 +69,8 @@
                 'cpu_runmode': 'ppc64_cpu --run-mode',
                 'cpu_freq': 'ppc64_cpu --frequency',
                 'cpu_dscr': 'ppc64_cpu --dscr',
- 'nvram': 'cat /dev/nvram',
             })
+ add_tar(report, '/dev/nvram', 'nvram.tar')
         attach_file_if_exists(report, '/var/log/platform')

     if ispSeries and not isPowerKVM:

------- Comment From <email address hidden> 2014-10-16 04:15 EDT-------
Still not fixed with latest builds.

root@lep8d:~# dpkg -l | grep apport
iF apport 2.14.7-0ubuntu7 all automatically generate crash reports for debugging
ii apport-gtk 2.14.7-0ubuntu6 all GTK+ frontend for the apport crash report system
ii apport-symptoms 0.20 all symptom scripts for apport
ii python-apport 2.14.7-0ubuntu6 all Python library for Apport crash report handling
ii python3-apport 2.14.7-0ubuntu7 all Python 3 library for Apport crash report handling
root@lep8d:~# apport-cli -f -p linux-generic --save=/home/sachin/linux.apport

*** Collecting problem information

The collected information can be sent to the developers to improve the
application. This might take a few minutes.
...tar: Removing leading `/' from member names
....ERROR: hook /usr/share/apport/general-hooks/powerpc.py crashed:
Traceback (most recent call last):
File "/usr/lib/python3/dist-packages/apport/report.py", line 197, in _run_hook
symb['add_info'](report, ui)
File "/usr/share/apport/general-hooks/powerpc.py", line 72, in add_info
'nvram': 'cat /dev/nvram',
File "/usr/lib/python3/dist-packages/apport/hookutils.py", line 469, in attach_root_command_outputs
buf = f.read().strip()
File "/usr/lib/python3.4/codecs.py", line 313, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb5 in position 1: invalid start byte
...
root@lep8d:~#

Martin, this appears to be a regression introduced by the hook added for bug #1336462. Could you check if you think the proposed patch is suitable?

Sorry for the late answer, this fell through the cracks. Did you actually test that this works? I. e. can a normal user actually read /dev/nvram? Currently the "cat /dev/nvram" is ran as root via attach_root_command_outputs(), and requiring root was the main reason why this "cat" approach was taken in the first place.

Does /dev/nvram actually contain text information and is this a locale/encoding problem? Or is it binary data, and unrelated to encodings?

With 14.10's apport, instead of cat the hook could call "base64 /dev/nvram" or a similar command which provides plaintext. But otherwise I think it would be better to fix attach_root_command_outputs() to get along with binary outputs.

> One dummy question remaining, how getting the tar file from the ascii output file if we need to ?

You can use "apport-unpack" to decode the apport report file into individual files (named by keys), and then just use tar to extract them further.

Fixed in trunk in http://bazaar.launchpad.net/~apport-hackers/apport/trunk/revision/2890

I tested it with /dev/nvram on amd64 (I just saw that this also exists there) and it works now. If you want you can apply this simple fix inline in /usr/lib/python3/dist-packages/apport/hookutils.py for testing, but I'll upload a stable release update for this too.

SRU uploaded, this now needs the SRU team to review/approve. Could you please test the package in -proposed once it arrives, to verify? Thanks!

This bug was fixed in the package apport - 2.15.1-0ubuntu2

---------------
apport (2.15.1-0ubuntu2) vivid; urgency=medium

  * Add systemd units for apport-noui.
  * Merge from trunk:
    - hookutils, attach_root_command_outputs(): Fix UnicodeDecodeError crash
      for non-textual values. (LP: #1370259)
 -- Martin Pitt <email address hidden> Thu, 08 Jan 2015 17:33:12 +0100

Hello bugproxy, or anyone else affected,

Accepted apport into utopic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.14.7-0ubuntu8.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

------- Comment From <email address hidden> 2015-01-12 08:59 EDT-------
Latest apport package works correctly.

root@ubuntu:~# apport-cli -f -p linux-generic --save=/home/linux.apport

*** Collecting problem information

The collected information can be sent to the developers to improve the
application. This might take a few minutes.
..........................................tar: Removing leading `/' from member names
.........

root@ubuntu:~# dpkg -l | grep apport
ii apport 2.14.7-0ubuntu8.1 all automatically generate crash reports for debugging
ii apport-gtk 2.14.7-0ubuntu8 all GTK+ frontend for the apport crash report system
ii apport-symptoms 0.20 all symptom scripts for apport
ii python3-apport 2.14.7-0ubuntu8.1 all Python 3 library for Apport crash report handling
root@ubuntu:~#

Thanks for testing!

This bug was fixed in the package apport - 2.14.7-0ubuntu8.1

---------------
apport (2.14.7-0ubuntu8.1) utopic-proposed; urgency=medium

  * hookutils, attach_root_command_outputs(): Fix UnicodeDecodeError crash for
    non-textual values. Cherry-picked from trunk r2890. (LP: #1370259)
 -- Martin Pitt <email address hidden> Thu, 08 Jan 2015 11:23:53 +0100

The verification of the Stable Release Update for apport has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.