[regression] unit test failures after Qt update

It appears that QUrl::topLevelDomain() now always returns an empty string.

This problem, which was picked up by Qt's own unit tests when building unmofified 5.5.1+dfsg-14ubuntu1, is documented in bug #1548686 and seems to come from something beneath Qt.

Similar to bug #1548686.

I just downgraded qt5 packages on my laptop from 5.5.1+dfsg-14ubuntu2 to 5.5.1+dfsg-14ubuntu1 (no other packages were touched), and the issue goes away.

qIsEffectiveTLD() does a lookup on a well-known list of suffixes generated from https://publicsuffix.org/list/public_suffix_list.dat.
The generated table in Qt’s source tree looks fine, and in fact it’s identical between those two versions of Qt.

I copied the relevant bits of the Qt source code that implement qTopLevelDomain() (including the generated table in qurltlds_p.h) and rebuilt it as a standalone test program on my machine, and it seems to be working fine. Next step is to do a full rebuild of Qt to compare.

Attaching the test code with relevant bits of Qt source code copied for future reference. To test, unpack in a directory, then run `qmake` then `make`, and execute the "test" binary.

On my x86-64 laptop running fully up-to-date xenial, I’m getting the following output:

    $ ./test
    QT: QUrl("http://example.org/foobar") "example.org" ""
    local: QUrl("http://example.org/foobar") "example.org" ".org"

I tried installing the packages from silo 51 (5.5.1+dfsg-14ubuntu3~xenial1~test1, which is a recent full rebuild), they don’t solve the issue.

My gdb-fu is weak and I haven’t managed to step into qTopLevelDomain() (other than in machine instructions, which I cannot make sense of).

I finally managed to use gdb to step into Qt code, and what I’m seeing in qTopLevelDomain() is that for e.g. "google.com" the sections are correctly extracted as ["google", "com"], and the for loop calls into qIsEffectiveTLD() for "com" (which should return true because "com" is part of the list at https://publicsuffix.org/list/public_suffix_list.dat), which itself calls into containsTLDEntry(), which computes a chunk index and then compares the entry to entries in that chunk. "com" is being compared to two entries: "accident-prevention.aero" and "net.ag". Those two entries are far apart in the generated table, so this doesn’t make much sense. It looks like somehow the initial computation of the chunk in containsTLDEntry() is broken.

When stepping into containsTLDEntry(), I’m seeing a value of 267 for index.

When I compute that value separately (entry="com", qt_hash(entry)=27229, tldCount=7150), I get a value of 5779.

I just found out that the qtbase-opensource-src debian package regenerates src/corelib/io/qurltlds_p.h at build time from /usr/share/publicsuffix/effective_tld_names.dat, which is provided by the publicsuffix package.

Looking at the contents of /usr/include/x86_64-linux-gnu/qt5/QtCore/5.5.1/QtCore/private/qurltlds_p.h (provided by qtbase5-private-dev), they widely differ from the version in the source tree, and indeed the "com" suffix isn’t in there, so that file is clearly corrupted.

I've tried to build qtbase against older versions of a) gccgo-6 (libgcc1), b) gcc5, c) glibc, without change in behavior.

I've now been building against older gnutls28 and publicsuffix, of which the latter indeed could affect this.

Ok it seems like a grep update on 2016-02-17 broke other things too.

Fixing via my guess that is apparently also the correct solution.

Bug #1547466

This turns out to be a regression in grep (bug #1547466). Building with a UTF-8 locale doesn’t exhibit the problem, but building with LANG=C LC_ALL=C does.

This bug was fixed in the package qtbase-opensource-src - 5.5.1+dfsg-14ubuntu3

---------------
qtbase-opensource-src (5.5.1+dfsg-14ubuntu3) xenial; urgency=medium

  * debian/patches/Xcb-fix-crash-on-screen-power-save.patch:
    - Fix additional cases where crash could occur on monitor disconnections
      or power save. (LP: #1551122)
  * Build with C.UTF-8 locale to (LP: #1551145) and re-enable tests (LP: #1548686)

 -- Timo Jyrinki <email address hidden> Wed, 02 Mar 2016 10:28:04 +0000