Ubuntu
pyqt5 package

Error in conversion of some characters from QString to Python unicode

Bug #1654840 reported by Dmitry Shachnev
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
pyqt5 (Ubuntu)
Fix Released
Undecided
Unassigned
Trusty
Fix Released
Undecided
Unassigned

Bug Description

# Test Case
$ python3
>>> from PyQt5.QtGui import QTextDocument
>>> d = QTextDocument()
>>> d.setPlainText('÷ Σ')
>>> d.toPlainText()
'÷ £'

This should not happen. d.toPlainText() should return '÷ Σ'.

# Impact
The bug was initially reported upstream: https://www.riverbankcomputing.com/pipermail/pyqt/2015-March/035627.html.

Some time ago I got the same issue in ReText CI, which uses Ubuntu Trusty. The test failure was observed at https://travis-ci.org/retext-project/retext/builds/186945524. With the proposed upload, the failure disappears.

This can also affect other editors which allow to input unicode, like eric.

# Proposed Fix
The fix which I am uploading to -proposed is taken from this upstream changeset:

2014-04-15 Phil Thompson <email address hidden>

    * qpy/QtCore/qpycore_qstring.cpp:
    Fixed a bug in the conversion of QStrings with surrogate pairs.
    [16a9c19d08cf] <5.2-maint>

It is present in all releases since v5.3.

# Regression Potential
The patch is small, should not introduce any regressions. If there is a regression, it is likely to be in the area that converts QStrings to Python Unicode objects, with the edge cases being different length strings between ASCII, 16-bit code points and code points that require surrogate pairs.

See original description
Revision history for this message
Robie Basak (racb) wrote : Please test proposed package

Hello Dmitry, or anyone else affected,

Accepted pyqt5 into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/pyqt5/5.2.1+dfsg-1ubuntu2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in pyqt5 (Ubuntu Trusty):
status: New → Fix Committed
tags: added: verification-needed
Revision history for this message
Robie Basak (racb) wrote :

When testing for SRU verification, please could you also test round-tripping some other strings through setPlainText and toPlainText? For example, combinations of empty, one and more -length strings that contain ASCII, 16-bit code points and code points that require surrogate pairs?

description: updated
Revision history for this message
Dmitry Shachnev (mitya57) wrote :

Sorry for the delay.

# dpkg-query -W -f '${Version}\n' python3-pyqt5
5.2.1+dfsg-1ubuntu2
# LC_ALL=C.UTF-8 python3
Python 3.4.3 (default, Nov 17 2016, 01:08:31)
[GCC 4.8.4] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from PyQt5.QtGui import QTextDocument
>>> d = QTextDocument()
>>> tests = ['', '÷ Σ', '÷ ф', 'Σ', '÷ $', '÷ €', '÷ \U00010437', '÷ \U00024B62', 'ascii', '€' * 1025, 'ちりぬるを', 'ᐊᓕᒍᖅ', 'ฉันกินกระจกได้']
>>> for string in tests:
... d.setPlainText(string)
... assert d.toPlainText() == string
>>> # all asserts passed

tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package pyqt5 - 5.2.1+dfsg-1ubuntu2

---------------
pyqt5 (5.2.1+dfsg-1ubuntu2) trusty; urgency=medium

  * Backport a change from upstream 5.2-maint branch to fix conversion
    of QStrings with surrogate pairs (fix_qstring_conversion.patch).
    Fixes LP: #1654840.

 -- Dmitry Shachnev <email address hidden> Sun, 08 Jan 2017 13:56:58 +0300

Changed in pyqt5 (Ubuntu Trusty):
status: Fix Committed → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of the Stable Release Update for pyqt5 has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.
Subscribing...

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.