Ubuntu
python-launchpadlib package

[py3] corrupts binary bug attachments

Bug #1425575 reported by Martin Pitt
16
This bug affects 3 people
Affects Status Importance Assigned to Milestone
python-launchpadlib (Ubuntu)
Confirmed
High
Colin Watson

Bug Description

While porting Apport to python3-launchpadlib, I noticed that python3-launchpadlib (or one of its underlying libraries) corrupts binary attachments. The attached reproducer attaches the gzip compressed value b'\x80\xFF' * 300 to some bug.

With python 2, it works fine, and you get a CoreDump.gz attachment that is 37 bytes. You can download it, and it looks as expected:

$ hd /tmp/CoreDump.gz
00000000 1f 8b 08 08 00 00 00 00 02 ff 43 6f 72 65 44 75 |..........CoreDu|
00000010 6d 70 00 6b f8 df 30 0a 47 21 d5 21 00 1c cd 1d |mp.k..0.G!.!....|
00000020 0a 58 02 00 00 |.X...|
00000025

But with python 3, you get a CoreDump.gz attachment which is 39 bytes and is corrupted:

$ zcat /tmp/CoreDump-py3.gz |hd

gzip: /tmp/CoreDump-py3.gz: invalid compressed data--crc error
00000000 80 ff 80 ff 80 ff 80 ff 80 ff 80 ff 80 ff 80 ff |................|

gzip: /tmp/CoreDump-py3.gz: invalid compressed data--length error
*
00000258

The compressed file has two extra 0A bytes:

$ diff -u <(hd /tmp/CoreDump.gz) <(hd /tmp/CoreDump-py3.gz)
--- /dev/fd/63 2015-02-25 17:00:22.336527519 +0100
+++ /dev/fd/62 2015-02-25 17:00:22.336527519 +0100
@@ -1,4 +1,4 @@
 00000000 1f 8b 08 08 00 00 00 00 02 ff 43 6f 72 65 44 75 |..........CoreDu|
-00000010 6d 70 00 6b f8 df 30 0a 47 21 d5 21 00 1c cd 1d |mp.k..0.G!.!....|
-00000020 0a 58 02 00 00 |.X...|
-00000025
+00000010 6d 70 00 6b f8 df 30 0a 47 21 d5 21 00 1c 0a cd |mp.k..0.G!.!....|
+00000020 1d 0a 0a 58 02 00 00 |...X...|
+00000027

Note that this needs to log into Launchpad, so you might want to adjust the credentials_file argument (the reproducer just uses apport's standard path). Also note that logging into Launchpad is currently broken with py3 (bug 1422249), so run it with python3 first to log in.

ProblemType: Bug
DistroRelease: Ubuntu 15.04
Package: python3-launchpadlib 1.10.3-1
ProcVersionSignature: Ubuntu 3.18.0-13.14-generic 3.18.5
Uname: Linux 3.18.0-13-generic x86_64
ApportVersion: 2.16.1-0ubuntu2
Architecture: amd64
CurrentDesktop: Unity
Date: Wed Feb 25 16:58:13 2015
EcryptfsInUse: Yes
PackageArchitecture: all
SourcePackage: python-launchpadlib
UpgradeStatus: No upgrade log present (probably fresh install)

Revision history for this message
Martin Pitt (pitti) wrote :
Changed in python-launchpadlib (Ubuntu):
assignee: nobody → Dimitri John Ledkov (xnox)
Dimitri John Ledkov (xnox)
Changed in python-launchpadlib (Ubuntu):
assignee: Dimitri John Ledkov (xnox) → Colin Watson (cjwatson)
Revision history for this message
Launchpad Janitor (janitor) wrote :

Status changed to 'Confirmed' because the bug affects multiple users.

Changed in python-launchpadlib (Ubuntu):
status: New → Confirmed
Revision history for this message
Scott Moser (smoser) wrote :

I've poked at this some, originally assuming there was some silly
decode() in a python3 path that was losing data, but I did not see it.
In both python2 and python3 we get to Http._request (in httplib2/__init__.py).
with the same body (minus python3 bytes, python2 string).

To break there, add 'import pdb; pdb.set_trace()' and then:
  import httplib2
  break httplib2.Http._request

You'll see at that point that py3 has bytes in its body and py2 and py3
headers are below:

python2
{'Authorization': 'OAuth realm="OAuth <snip> LHZx0NKGQ4g5S84Dn5mfmFR"',
 'accept': 'application/json',
 'accept-encoding': 'gzip, deflate',
 'content-type': 'multipart/form-data; boundary="===============4087956070816413326=="',
 'user-agent': u'lazr.restfulclient 0.14.0; application="test"; oauth_consumer="System-wide: Ubuntu (milhouse)"'}

python3
{b'Authorization': b'OAuth realm="OAuth <snip> GQ4g5S84Dn5mfmFR"',
 'accept': 'application/json',
 'accept-encoding': 'gzip, deflate',
 'content-type': 'multipart/form-data; boundary="===============5370337788628842176=="',
 'user-agent': 'lazr.restfulclient 0.14.0; application="test"; '
               'oauth_consumer="System-wide: Ubuntu (milhouse)"'}

I'm not sure where it is going wrong from that point.

Changed in python-launchpadlib (Ubuntu):
importance: Undecided → High
Revision history for this message
Scott Moser (smoser) wrote :

set this as 'High' at least per ubuntu guidelines, there isn't really a workaround other than "use python2". :-(

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.