resquashfs test fails if snap has symlinks

Bug #1555305 reported by Jamie Strandboge
26
This bug affects 5 people
Affects Status Importance Assigned to Milestone
click-reviewers-tools (Ubuntu)
Invalid
High
Unassigned
squashfs-tools (Ubuntu)
Fix Released
High
Tyler Hicks
Trusty
Fix Released
High
Tyler Hicks
Xenial
Fix Released
High
Tyler Hicks
Zesty
Fix Released
High
Tyler Hicks
Artful
Fix Released
High
Tyler Hicks
Bionic
Fix Released
High
Tyler Hicks

Bug Description

[Impact]

 * The unsquashfs utility does not preserve atime and mtime timestamps
   of symlink inodes when unpacking squashfs images.

 * This makes it impossible to use a combination of unsquashfs and
   mksquashfs to sanity check snaps uploaded to the snap store since
   the resulting snap will not match the original snap due to symlink
   times being set to the current time when unsquashfs was executed.

[Test Case]

 * Extract the test image:
   $ unsquashfs -d lp1555305 lp1555305.squashfs

 * Check the timestamp of the "link" file
   $ ls -al lp1555305/link

 * Incorrect output (timestamp will be current time):
   lrwxrwxrwx 1 tyhicks tyhicks 4 Dec 11 22:05 link -> file

 * Expected output:
   lrwxrwxrwx 1 tyhicks tyhicks 4 Aug 25 1991 /tmp/lp1555305/link -> file

[Regression Potential]

 * The regression potential is small as long as there is no software
   out there that depends on the behavior of setting symlink timestamps
   to the current time. I highly doubt that there is anything out there
   that relies on that behavior but there's always a chance.

[Original Descipription]

Create the files:
$ mkdir test
$ cd test
$ touch foo
$ ln -s foo bar
$ mkdir meta
$ cat >meta/snap.yaml <<EOM
name: test
version: 1.0
architectures: [all]
EOM

Create the initial snap
$ cd ..
$ snapcraft snap ./test
...
Snapped test_1.0_all.snap

Unsquash the snap:
$ unsquashfs -d unsquashed test_1.0_all.snap
Parallel unsquashfs: Using 4 processors
3 inodes (2 blocks) to write

[=======================================================================|] 2/2 100%

created 2 files
created 2 directories
created 1 symlinks
created 0 devices
created 0 fifos

Resquash the snap (the sleep is to make sure the time changes on the symlink):
$ unsquashfs -fstime ./test_1.0_all.snap
1457551603
$ mksquashfs ./unsquashed ./test_1.0_all.snap.repacked -fstime 1457551603 -noappend -comp xz -all-root -no-xattrs
...

Compare:
$ sha512sum test_1.0_all.snap
40a15430503783541b362942491ee614b16941676d6c3edd86005af7b6b0782374e41c0c006dccb3a164d2c0270ace66639fcdee5ece3571db1c7544d59030e3 test_1.0_all.snap

$ sha512sum test_1.0_all.snap.repack
d8b9626337b8f09288f6aab675041dba639523437dd1d8f11b3f883cfa73ea434f2be06f5cd9cdec4ac9bdf2a40f8b484dc516ec851870c81be133cbdebf35a4 test_1.0_all.snap.repack

Removing the symlink makes the check work.

Changed in click-reviewers-tools (Ubuntu):
importance: Undecided → High
description: updated
Changed in click-reviewers-tools (Ubuntu):
status: New → Confirmed
Revision history for this message
Tyler Hicks (tyhicks) wrote :

Fix submitted to the upstream squashfs project:

 https://sourceforge.net/p/squashfs/mailman/message/36140758/

Revision history for this message
Tyler Hicks (tyhicks) wrote :

Marking the click-reviewers-tools task as invalid since the bug fix is in squashfs-tools and there's no changes needed for click-reviewers-tools.

Changed in click-reviewers-tools (Ubuntu):
status: Confirmed → Invalid
no longer affects: click-reviewers-tools (Ubuntu Trusty)
no longer affects: click-reviewers-tools (Ubuntu Xenial)
no longer affects: click-reviewers-tools (Ubuntu Zesty)
no longer affects: click-reviewers-tools (Ubuntu Artful)
no longer affects: click-reviewers-tools (Ubuntu Bionic)
Changed in squashfs-tools (Ubuntu Trusty):
importance: Undecided → High
Changed in squashfs-tools (Ubuntu Xenial):
importance: Undecided → High
Changed in squashfs-tools (Ubuntu Zesty):
importance: Undecided → High
Changed in squashfs-tools (Ubuntu Artful):
importance: Undecided → High
Changed in squashfs-tools (Ubuntu Bionic):
importance: Undecided → High
Changed in squashfs-tools (Ubuntu Trusty):
assignee: nobody → Tyler Hicks (tyhicks)
Changed in squashfs-tools (Ubuntu Xenial):
assignee: nobody → Tyler Hicks (tyhicks)
Changed in squashfs-tools (Ubuntu Zesty):
assignee: nobody → Tyler Hicks (tyhicks)
Changed in squashfs-tools (Ubuntu Artful):
assignee: nobody → Tyler Hicks (tyhicks)
Changed in squashfs-tools (Ubuntu Bionic):
assignee: nobody → Tyler Hicks (tyhicks)
status: New → In Progress
Changed in squashfs-tools (Ubuntu Artful):
status: New → In Progress
Changed in squashfs-tools (Ubuntu Zesty):
status: New → In Progress
Changed in squashfs-tools (Ubuntu Xenial):
status: New → In Progress
Changed in squashfs-tools (Ubuntu Trusty):
status: New → In Progress
Revision history for this message
Tyler Hicks (tyhicks) wrote :

I'm attaching a squashfs image containing a symlink named "link" and a regular file named "file". The symlink has a timestamp of "25 Aug 91 20:57:08 GMT".

description: updated
Tyler Hicks (tyhicks)
description: updated
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package squashfs-tools - 1:4.3-4ubuntu2

---------------
squashfs-tools (1:4.3-4ubuntu2) bionic; urgency=medium

  * debian/patches/0005-add-fstime.patch: Fix -Wint-conversion warning by
    initializing the time_t variable with (time_t)-1 instead of NULL
  * debian/patches/0008-unsquashfs-preserve-symlink-times.patch: Preserve
    atime and mtime of symlink inodes in unsquashfs rather than using the
    current time (LP: #1555305)

 -- Tyler Hicks <email address hidden> Mon, 11 Dec 2017 20:56:21 +0000

Changed in squashfs-tools (Ubuntu Bionic):
status: In Progress → Fix Released
Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Jamie, or anyone else affected,

Accepted squashfs-tools into artful-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/squashfs-tools/1:4.3-4ubuntu1.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-artful to verification-done-artful. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-artful. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in squashfs-tools (Ubuntu Artful):
status: In Progress → Fix Committed
tags: added: verification-needed verification-needed-artful
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Jamie, or anyone else affected,

Accepted squashfs-tools into zesty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/squashfs-tools/1:4.3-3ubuntu2.17.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-zesty to verification-done-zesty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-zesty. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in squashfs-tools (Ubuntu Zesty):
status: In Progress → Fix Committed
tags: added: verification-needed-zesty
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Jamie, or anyone else affected,

Accepted squashfs-tools into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/squashfs-tools/1:4.3-3ubuntu2.16.04.1 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in squashfs-tools (Ubuntu Xenial):
status: In Progress → Fix Committed
tags: added: verification-needed-xenial
Revision history for this message
Brian Murray (brian-murray) wrote :

Hello Jamie, or anyone else affected,

Accepted squashfs-tools into trusty-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/squashfs-tools/1:4.2+20130409-2ubuntu0.14.04.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-trusty to verification-done-trusty. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-trusty. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in squashfs-tools (Ubuntu Trusty):
status: In Progress → Fix Committed
tags: added: verification-needed-trusty
Revision history for this message
Tyler Hicks (tyhicks) wrote :

I've successfully performed the test described in the [Test Case] section on artful, zesty, xenial, and trusty in both amd64 and i386 VMs. I see that there are some autopkgtest failures/errors so I'll investigate those before marking these as verification-done.

Revision history for this message
Tyler Hicks (tyhicks) wrote :

* Failing Xenial snapd tests
- autopkgtest:ubuntu-16.04-amd64:tests/main/completion fails with and without the squashfs-tools in xenial-proposed
- autopkgtest:ubuntu-16.04-amd64:tests/main/searching is attempting to find the "cassandra" snap in the database section but I can confirm that it isn't listed in the database section of the store
- s390x: No tests are ever ran due to the tests requiring "machine-level isolation" but that not being available on s390x. However, snapd s390x test runs have been hitting an error for about the last month.

* Failing Xenial snapcraft tests
- test_catkin_part_builds_after_python_part() is failing on all snapcraft test runs

From what I can tell, the snapd and snapcraft test failures on Xenial are not due to the squashfs-tools changes.

tags: added: verification-done-xenial
removed: verification-needed-xenial
Revision history for this message
Tyler Hicks (tyhicks) wrote :

The failing Trusty snapd tests are marked as "Always failed" and are unrelated to the squashfs-tools changes:

  fatal: unable to access 'https://github.com/kardianos/govendor/': Failed to connect to github.com port 443: Connection timed out

tags: added: verification-done-trusty
removed: verification-needed-trusty
Revision history for this message
Tyler Hicks (tyhicks) wrote :

* Failing Zesty and Artful snapd tests
- autopkgtest:ubuntu-16.04-amd64:tests/main/completion fails with and without the squashfs-tools in xenial-proposed
- autopkgtest:ubuntu-16.04-amd64:tests/main/searching is attempting to find the "cassandra" snap in the database section but I can confirm that it isn't listed in the database section of the store
- s390x: No tests are ever ran due to the tests requiring "machine-level isolation" but that not being available on s390x. However, snapd s390x test runs have been hitting an error for about the last month.

* Failing Zesty and Artful snapcraft tests
- These have all been failing recently

* Failed Zesty and Artful lxd tests on s390x
- These tests have only ever passed with "SKIP Test requires machine-level isolation but testbed does not provide that" but they've recently started running and failing every run

From what I can tell, the snapd, snapcraft, and lxd test failures on Zesty and Artful are not due to the squashfs-tools changes.

tags: added: verification-done-artful verification-done-zesty
removed: verification-needed-artful verification-needed-zesty
tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package squashfs-tools - 1:4.3-4ubuntu1.1

---------------
squashfs-tools (1:4.3-4ubuntu1.1) artful; urgency=medium

  * debian/patches/0008-unsquashfs-preserve-symlink-times.patch: Preserve
    atime and mtime of symlink inodes in unsquashfs rather than using the
    current time (LP: #1555305)

 -- Tyler Hicks <email address hidden> Mon, 11 Dec 2017 20:51:14 +0000

Changed in squashfs-tools (Ubuntu Artful):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for squashfs-tools has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package squashfs-tools - 1:4.3-3ubuntu2.17.04.1

---------------
squashfs-tools (1:4.3-3ubuntu2.17.04.1) zesty; urgency=medium

  * debian/patches/0007-unsquashfs-preserve-symlink-times.patch: Preserve
    atime and mtime of symlink inodes in unsquashfs rather than using the
    current time (LP: #1555305)

 -- Tyler Hicks <email address hidden> Mon, 11 Dec 2017 21:02:54 +0000

Changed in squashfs-tools (Ubuntu Zesty):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package squashfs-tools - 1:4.3-3ubuntu2.16.04.1

---------------
squashfs-tools (1:4.3-3ubuntu2.16.04.1) xenial; urgency=medium

  * debian/patches/0007-unsquashfs-preserve-symlink-times.patch: Preserve
    atime and mtime of symlink inodes in unsquashfs rather than using the
    current time (LP: #1555305)

 -- Tyler Hicks <email address hidden> Mon, 11 Dec 2017 21:02:07 +0000

Changed in squashfs-tools (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package squashfs-tools - 1:4.2+20130409-2ubuntu0.14.04.2

---------------
squashfs-tools (1:4.2+20130409-2ubuntu0.14.04.2) trusty; urgency=medium

  * debian/patches/0003-unsquashfs-preserve-symlink-times.patch: Preserve
    atime and mtime of symlink inodes in unsquashfs rather than using the
    current time (LP: #1555305)

 -- Tyler Hicks <email address hidden> Mon, 11 Dec 2017 21:06:30 +0000

Changed in squashfs-tools (Ubuntu Trusty):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.