duplicate signature for Package problems can be too short

Bug #1692127 reported by Brian Murray
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
apport (Ubuntu)
Fix Released
Undecided
Unassigned
Xenial
Fix Released
Undecided
Brian Murray
Yakkety
Fix Released
Undecided
Brian Murray
Zesty
Fix Released
High
Brian Murray
Artful
Fix Released
Undecided
Unassigned

Bug Description

[Impact]
apport is creating a shorter than appropriate duplicate signature in apport-package bug reports which can lead to misconsolidation and confused developers.

[Test Case]
1) Boot a system with an old kernel installed or a freshly installed system
2) edit /etc/default/grub and replace a ` with a '
3) Upgrade the kernel, watch it crash
4) Discover a crash report with a DuplicateSignature whose 2nd line starts with "Examining"

With the version of the package from -proposed the DuplicateSignature in this case should really have a 2nd line that starts with 'Setting up linux-image* ...'. While in this particular case not much is lost in other cases valuable information will be missing.

[Regression Potential]
The code change is correct and has been tested in artful, additionally I'm using the same regular expression to clean up apport-package reports as they come in to LP. The only risk here is a typo in the patch that would cause the hook (the change is in the ubuntu general hook) to error out which apport would handle gracefully and we just wouldn't have a DuplicateSignature.

[The Original Description]
Steve brought up bug 1691983 and how it's duplicate signature is missing information. It contains:

package:shim-signed:1.27~16.04.1+0.9+1474479173.6c180c6-1ubuntu1
Installing for x86_64-efi platform.
Installation finished. No error reported.
Running in non-interactive mode, doing nothing.
dpkg: error processing package shim-signed (--configure):
 subprocess installed post-installation script returned error exit status 1

But is missing:

Setting up shim-signed (1.27~16.04.1+0.9+1474479173.6c180c6-1ubuntu1) ...
locale: Cannot set LC_CTYPE to default locale: No such file or directory
locale: Cannot set LC_MESSAGES to default locale: No such file or directory
locale: Cannot set LC_ALL to default locale: No such file or directory
debconf: unable to initialize frontend: Passthrough
debconf: (Cannot connect to /tmp/aptdaemon-7jrqb_du/debconf.socket: Connection refused at (eval 18) line 3.)
debconf: falling back to frontend: Noninteractive

This is because of the following code in data/general-hooks/ubuntu.py:

145 PKG_MSGS = ('Authenticating', 'De-configuring', 'Examining',
146 'Installing ', 'Preparing', 'Processing triggers', 'Purging',
147 'Removing', 'Replaced', 'Replacing', 'Setting up',
148 'Unpacking', 'Would remove')
149 for line in termlog.split('\n'):
150 if line.startswith(PKG_MSGS):
151 dupe_sig = '%s\n' % line
152 continue

The shim-signed package prints a line starts with "Installing " and that matches PKG_MSGS so we reset the dupe_sig because that's a message we'd expect from a package manager.

Revision history for this message
Brian Murray (brian-murray) wrote :

We could continue if the line equals "Installing for x86_64-efi platform." but that would only work for shim-signed and seems fragile.

tags: added: artful xenial yakkety zesty
Revision history for this message
Steve Langasek (vorlon) wrote :

When each of these messages are output by dpkg, they also match the pattern '\.\.\.[[:space:]]*$'. So perhaps that should be included as part of the filter?

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apport - 2.20.5-0ubuntu4

---------------
apport (2.20.5-0ubuntu4) artful; urgency=medium

  * data/general-hooks/ubuntu.py: Modify how a duplicate signature is created
    for package installation failures. (LP: #1692127)

 -- Brian Murray <email address hidden> Mon, 22 May 2017 16:37:50 -0700

Changed in apport (Ubuntu Artful):
status: New → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote :

This should be SRU'ed to stable releases but let's make sure it is working properly first.

Changed in apport (Ubuntu Zesty):
status: New → Triaged
importance: Undecided → High
assignee: nobody → Brian Murray (brian-murray)
Revision history for this message
Brian Murray (brian-murray) wrote :

Trusty uses the old method of creating a duplicate signature so does not require this fix.

104 # build a duplicate signature tag for package reports
105 if report.get('ProblemType') == 'Package' and 'ErrorMessage' in report and 'Package' in report:
106 (package, version) = report['Package'].split(None, 1)
107 dupe_sig = 'package:%s:%s:%s' % (package, version, report['ErrorMessage'])
108 report['DuplicateSignature'] = dupe_sig

Changed in apport (Ubuntu Yakkety):
status: New → Triaged
Changed in apport (Ubuntu Xenial):
status: New → Triaged
description: updated
Changed in apport (Ubuntu Yakkety):
assignee: nobody → Brian Murray (brian-murray)
Changed in apport (Ubuntu Xenial):
assignee: nobody → Brian Murray (brian-murray)
status: Triaged → In Progress
Changed in apport (Ubuntu Yakkety):
status: Triaged → In Progress
Changed in apport (Ubuntu Zesty):
status: Triaged → In Progress
Revision history for this message
Steve Langasek (vorlon) wrote : Please test proposed package

Hello Brian, or anyone else affected,

Accepted apport into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.4-0ubuntu4.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

description: updated
Changed in apport (Ubuntu Zesty):
status: In Progress → Fix Committed
tags: added: verification-needed
Changed in apport (Ubuntu Yakkety):
status: In Progress → Fix Committed
Revision history for this message
Steve Langasek (vorlon) wrote :

Hello Brian, or anyone else affected,

Accepted apport into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.3-0ubuntu8.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in apport (Ubuntu Xenial):
status: In Progress → Fix Committed
Revision history for this message
Steve Langasek (vorlon) wrote :

Hello Brian, or anyone else affected,

Accepted apport into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.1-0ubuntu2.7 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Revision history for this message
Brian Murray (brian-murray) wrote :

The upload of apport which fixed this bug did not cause but revealed an autopkgtest failure with test_signal_crashes.py in apport. The tests test_core_dump_packaged and test_core_dump_unpackaged are failing because the core file being produced is greater than the ulimit set for core files. This limit is arbitrary and I've bumped it from 1 million to 10 million bytes to allow the test pass.

Revision history for this message
Łukasz Zemczak (sil2100) wrote : Change of SRU verification policy

As part of a recent change in the Stable Release Update verification policy we would like to inform that for a bug to be considered verified for a given release a verification-done-$RELEASE tag needs to be added to the bug where $RELEASE is the name of the series the package that was tested (e.g. verification-done-xenial). Please note that the global 'verification-done' tag can no longer be used for this purpose.

Thank you!

Revision history for this message
Adam Conrad (adconrad) wrote : Please test proposed package

Hello Brian, or anyone else affected,

Accepted apport into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.1-0ubuntu2.8 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Revision history for this message
Adam Conrad (adconrad) wrote :

Hello Brian, or anyone else affected,

Accepted apport into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.3-0ubuntu8.5 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Revision history for this message
Adam Conrad (adconrad) wrote :

Hello Brian, or anyone else affected,

Accepted apport into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.4-0ubuntu4.3 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Revision history for this message
Steve Langasek (vorlon) wrote :

Hello Brian, or anyone else affected,

Accepted apport into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.4-0ubuntu4.4 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-zesty to verification-done-zesty.If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-zesty. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-needed-zesty
tags: added: verification-needed-yakkety
Revision history for this message
Steve Langasek (vorlon) wrote :

Hello Brian, or anyone else affected,

Accepted apport into yakkety-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.3-0ubuntu8.6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-yakkety to verification-done-yakkety.If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-yakkety. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

tags: added: verification-needed-xenial
Revision history for this message
Steve Langasek (vorlon) wrote :

Hello Brian, or anyone else affected,

Accepted apport into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/apport/2.20.1-0ubuntu2.9 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial.If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Revision history for this message
Brian Murray (brian-murray) wrote :

This worked for me in zesty with apport version 2.20.4-0ubuntu4.4:

DuplicateSignature:
 package:linux-image-4.10.0-22-generic:4.10.0-22.24
 Setting up linux-image-4.10.0-22-generic (4.10.0-22.24) ...
 Running depmod.
 update-initramfs: deferring update (hook will be called later)
 Not updating initrd symbolic links since we are being updated/reinstalled
 (4.10.0-22.24 was configured last, according to dpkg)
 Not updating image symbolic links since we are being updated/reinstalled
 (4.10.0-22.24 was configured last, according to dpkg)
 Examining /etc/kernel/postinst.d.
 run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.10.0-22-generic /boot/vmlinuz-4.10.0-22-generic
 run-parts: executing /etc/kernel/postinst.d/dkms 4.10.0-22-generic /boot/vmlinuz-4.10.0-22-generic
 run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.10.0-22-generic /boot/vmlinuz-4.10.0-22-generic
 update-initramfs: Generating /boot/initrd.img-4.10.0-22-generic
 run-parts: executing /etc/kernel/postinst.d/pm-utils 4.10.0-22-generic /boot/vmlinuz-4.10.0-22-generic
 run-parts: executing /etc/kernel/postinst.d/unattended-upgrades 4.10.0-22-generic /boot/vmlinuz-4.10.0-22-generic
 run-parts: executing /etc/kernel/postinst.d/update-notifier 4.10.0-22-generic /boot/vmlinuz-4.10.0-22-generic
 run-parts: executing /etc/kernel/postinst.d/zz-update-grub 4.10.0-22-generic /boot/vmlinuz-4.10.0-22-generic
 /usr/sbin/grub-mkconfig: 15: /etc/default/grub: Syntax error: Unterminated quoted string
 run-parts: /etc/kernel/postinst.d/zz-update-grub exited with return code 2
 Failed to process /etc/kernel/postinst.d at /var/lib/dpkg/info/linux-image-4.10.0-22-generic.postinst line 1052.
 dpkg: error processing package linux-image-4.10.0-22-generic (--configure):
  subprocess installed post-installation script returned error exit status 2

tags: added: verification-done-zesty
removed: verification-needed-zesty
Revision history for this message
Brian Murray (brian-murray) wrote :

This also works for me on yakkety with apport version 2.20.3-0ubuntu8.6:

Setting up linux-image-4.8.0-58-generic (4.8.0-58.63) ...
Running depmod.
update-initramfs: deferring update (hook will be called later)
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.8.0-58-generic /boot/vmlinuz-4.8.0-58-generic
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.8.0-58-generic /boot/vmlinuz-4.8.0-58-generic
update-initramfs: Generating /boot/initrd.img-4.8.0-58-generic
run-parts: executing /etc/kernel/postinst.d/pm-utils 4.8.0-58-generic /boot/vmlinuz-4.8.0-58-generic
run-parts: executing /etc/kernel/postinst.d/unattended-upgrades 4.8.0-58-generic /boot/vmlinuz-4.8.0-58-generic
run-parts: executing /etc/kernel/postinst.d/update-notifier 4.8.0-58-generic /boot/vmlinuz-4.8.0-58-generic
run-parts: executing /etc/kernel/postinst.d/zz-update-grub 4.8.0-58-generic /boot/vmlinuz-4.8.0-58-generic
/usr/sbin/grub-mkconfig: 15: /etc/default/grub: Syntax error: Unterminated quoted string
run-parts: /etc/kernel/postinst.d/zz-update-grub exited with return code 2
Failed to process /etc/kernel/postinst.d at /var/lib/dpkg/info/linux-image-4.8.0-58-generic.postinst line 1052.
dpkg: error processing package linux-image-4.8.0-58-generic (--configure):
 subprocess installed post-installation script returned error exit status 2

tags: added: verification-done-yakkety
removed: verification-needed-yakkety
Revision history for this message
Brian Murray (brian-murray) wrote :

This also works for me on xenial with apport version 2.20.1-0ubuntu2.9:

== DuplicateSignature =================================
package:linux-image-4.4.0-84-generic:4.4.0-84.107
Setting up linux-image-4.4.0-84-generic (4.4.0-84.107) ...
Running depmod.
update-initramfs: deferring update (hook will be called later)
Examining /etc/kernel/postinst.d.
run-parts: executing /etc/kernel/postinst.d/apt-auto-removal 4.4.0-84-generic /boot/vmlinuz-4.4.0-84-generic
run-parts: executing /etc/kernel/postinst.d/initramfs-tools 4.4.0-84-generic /boot/vmlinuz-4.4.0-84-generic
update-initramfs: Generating /boot/initrd.img-4.4.0-84-generic
run-parts: executing /etc/kernel/postinst.d/pm-utils 4.4.0-84-generic /boot/vmlinuz-4.4.0-84-generic
run-parts: executing /etc/kernel/postinst.d/unattended-upgrades 4.4.0-84-generic /boot/vmlinuz-4.4.0-84-generic
run-parts: executing /etc/kernel/postinst.d/update-notifier 4.4.0-84-generic /boot/vmlinuz-4.4.0-84-generic
run-parts: executing /etc/kernel/postinst.d/zz-update-grub 4.4.0-84-generic /boot/vmlinuz-4.4.0-84-generic
/usr/sbin/grub-mkconfig: 15: /etc/default/grub: Syntax error: Unterminated quoted string
run-parts: /etc/kernel/postinst.d/zz-update-grub exited with return code 2
Failed to process /etc/kernel/postinst.d at /var/lib/dpkg/info/linux-image-4.4.0-84-generic.postinst line 1052.
dpkg: error processing package linux-image-4.4.0-84-generic (--configure):
 subprocess installed post-installation script returned error exit status 2

tags: added: verification-done-xenial
removed: verification-needed verification-needed-xenial
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apport - 2.20.4-0ubuntu4.4

---------------
apport (2.20.4-0ubuntu4.4) zesty; urgency=medium

  * test/test_signal_crashes.py: delete the test which uses an arbitrary
    unpredictable core file size.

apport (2.20.4-0ubuntu4.3) zesty; urgency=medium

  * test/test_signal_crashes.py: a ulimit of 1M bytes isn't enough to produce
    a core file anymore so bump it to 10M.

apport (2.20.4-0ubuntu4.2) zesty; urgency=medium

  * data/general-hooks/ubuntu.py: Modify how a duplicate signature is created
    for package installation failures. (LP: #1692127)

 -- Brian Murray <email address hidden> Thu, 29 Jun 2017 11:41:12 -0700

Changed in apport (Ubuntu Zesty):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for apport has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apport - 2.20.3-0ubuntu8.6

---------------
apport (2.20.3-0ubuntu8.6) yakkety; urgency=medium

  * test/test_signal_crashes.py: delete the test which uses an arbitrary
    unpredictable core file size.

apport (2.20.3-0ubuntu8.5) yakkety; urgency=medium

  * test/test_signal_crashes.py: a ulimit of 1M bytes isn't enough to produce
    a core file anymore so bump it to 10M.

apport (2.20.3-0ubuntu8.4) yakkety; urgency=medium

  * data/general-hooks/ubuntu.py: Modify how a duplicate signature is created
    for package installation failures. (LP: #1692127)

 -- Brian Murray <email address hidden> Thu, 29 Jun 2017 13:30:39 -0700

Changed in apport (Ubuntu Yakkety):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package apport - 2.20.1-0ubuntu2.9

---------------
apport (2.20.1-0ubuntu2.9) xenial; urgency=medium

  * test/test_signal_crashes.py: delete the test which uses an arbitrary
    unpredictable core file size.

apport (2.20.1-0ubuntu2.8) xenial; urgency=medium

  * test/test_signal_crashes.py: a ulimit of 1M bytes isn't enough to produce
    a core file anymore so bump it to 10M.

apport (2.20.1-0ubuntu2.7) xenial; urgency=medium

  * data/general-hooks/ubuntu.py: Modify how a duplicate signature is created
    for package installation failures. (LP: #1692127)

 -- Brian Murray <email address hidden> Thu, 29 Jun 2017 13:33:50 -0700

Changed in apport (Ubuntu Xenial):
status: Fix Committed → Fix Released
tags: added: id-5956aefd1fbc2cddbec3dbc9
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.