Provide -root.tar.xz for bionic and later

Bug #1585233 reported by Scott Moser
74
This bug affects 13 people
Affects Status Importance Assigned to Milestone
cloud-images
Fix Released
High
Unassigned
livecd-rootfs (Ubuntu)
Fix Released
Undecided
Unassigned
Bionic
Fix Released
Undecided
Unassigned

Bug Description

Hi,
compared to xenial [1], yakkety [2] is missing a -root.tar.xz file.
The .tar.xz file is quite valuable compared to a -squashfs image for at least the following reasons:

a.) it is much less common denominator (mac or windows can open it)
b.) it is much smaller (138M compared to 171M for most recent xenial image)
c.) it can be "streamed", where the squashfs image must be downloaded in complete before it can be viewed. This is a significant optimization for for a system that can download and check while extracting.

[1] http://cloud-images.ubuntu.com/daily/server/xenial/current/
[2] http://cloud-images.ubuntu.com/daily/server/yakkety/current/

Related branches

Scott Moser (smoser)
description: updated
description: updated
description: updated
Dan Watkins (oddbloke)
description: updated
Dan Watkins (oddbloke)
Changed in cloud-images:
importance: Undecided → Wishlist
Scott Moser (smoser)
summary: - please provide -root.tar.xz for yakkety
+ please provide -root.tar.xz for artful
Revision history for this message
Scott Moser (smoser) wrote : Re: please provide -root.tar.xz for artful

We do not have a tar of any type available for artful.
I came across this pain twice in the past week.
a.) curtin would like to have a .tar file to install
   https://gist.github.com/smoser/375123ef1ef098be23cc856a5772c5c8
It could grow support for installing from a squashfs image, but tar just makes more sense.

b.) rooturl cloud-initramfs-rooturl makes more sense to use a .tar.gz
   root=http://.../some.tar.xz makes more sense than .squashfs.
   To use the squashfs, rooturl has to download the whole thing and then mount
it, and then because squashfs is not writable, we need to do overlayroot.

Thus, instead of:
   root=<url>.tar.xz

We have to do:
   root=<url>.squashfs overlayroot=tmpfs

I ran into this need when trying to test bug 1711760.

In summary, the .tar.xz (or .tar.gz) was just very useful and serves a purpose not served by other formats. Please bring it back.

Revision history for this message
Valery Tschopp (valery-tschopp) wrote :

Please provide -root.tar.xz for bionic, the OpenStack diskimage-builder [1] requires it to produce customized cloud images.

[1] https://github.com/openstack/diskimage-builder

Revision history for this message
Volodymyr Litovka (doka.ua) wrote :

I've patched diskimage-builder component for myself, patch is available -

http://lists.openstack.org/pipermail/openstack-operators/2018-February/014858.html

Note: I didn't check it for other versions except Artful.

Revision history for this message
Dan Watkins (oddbloke) wrote :

Hello all,

We've decided to reintroduce the tarball, and will be scheduling this for our next iteration.

Thanks,

Dan

Changed in cloud-images:
status: New → Confirmed
importance: Wishlist → High
summary: - please provide -root.tar.xz for artful
+ Provide -root.tar.xz for bionic and later
Adam Conrad (adconrad)
Changed in livecd-rootfs (Ubuntu):
status: New → In Progress
Adam Conrad (adconrad)
Changed in livecd-rootfs (Ubuntu):
status: In Progress → Fix Committed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package livecd-rootfs - 2.531

---------------
livecd-rootfs (2.531) cosmic; urgency=medium

  [ Daniel Watkins ]
  * ubuntu-cpc: Use linux-kvm for all amd64 minimized builds (LP: #1775472).

  [ Steve Langasek ]
  * live-build/ubuntu-cpc/hooks/033-disk-image-uefi.binary: don't manually
    install grub-efi-amd64 since that forces off grub-pc, which is otherwise
    now coinstallable.

 -- Steve Langasek <email address hidden> Wed, 06 Jun 2018 17:34:07 -0700

Changed in livecd-rootfs (Ubuntu):
status: Fix Committed → Fix Released
Revision history for this message
shatun (shatun) wrote :

Hi all,
Any chance this will be assigned and fixed any time soon?

Thanks

Revision history for this message
Dan Watkins (oddbloke) wrote : Re: [Bug 1585233] Re: Provide -root.tar.xz for bionic and later

Hi shatun,

On Sun, Jul 22, 2018 at 09:15:53AM -0000, shatun wrote:
> Any chance this will be assigned and fixed any time soon?

This work is in-progress at the moment; you can see the artifacts are
now being produced for cosmic[0]. The next step is a backport to
bionic, which I would anticipate starting this week.

Thanks for your interest!

Dan

[0] http://cloud-images.ubuntu.com/cosmic/current/

Revision history for this message
shatun (shatun) wrote :

Awesome, thanks for the update!

On Mon, Jul 23, 2018 at 4:35 PM Dan Watkins <email address hidden>
wrote:

> Hi shatun,
>
> On Sun, Jul 22, 2018 at 09:15:53AM -0000, shatun wrote:
> > Any chance this will be assigned and fixed any time soon?
>
> This work is in-progress at the moment; you can see the artifacts are
> now being produced for cosmic[0]. The next step is a backport to
> bionic, which I would anticipate starting this week.
>
> Thanks for your interest!
>
>
> Dan
>
>
> [0] http://cloud-images.ubuntu.com/cosmic/current/
>
> --
> You received this bug notification because you are subscribed to the bug
> report.
> https://bugs.launchpad.net/bugs/1585233
>
> Title:
> Provide -root.tar.xz for bionic and later
>
> To manage notifications about this bug go to:
> https://bugs.launchpad.net/cloud-images/+bug/1585233/+subscriptions
>

Revision history for this message
Dan Watkins (oddbloke) wrote :

Attached is the debdiff to reintroduce the tarball for bionic.

Revision history for this message
Robert C Jennings (rcj) wrote :

@daniel-thewatkins, please add '--xattrs' to the tar command in the root-xz binary hook ala https://code.launchpad.net/~vorlon/livecd-rootfs/lp.1302192/+merge/352055

Revision history for this message
Dan Watkins (oddbloke) wrote :

That change hasn't yet landed in the cosmic archive, so isn't yet suitable for SRU to bionic.

This new hook will get updated along with all the other tarball generation when the xattrs change is backported.

Revision history for this message
Brian Murray (brian-murray) wrote : Please test proposed package

Hello Scott, or anyone else affected,

Accepted livecd-rootfs into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd-rootfs/2.525.6 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed.Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

Changed in livecd-rootfs (Ubuntu Bionic):
status: New → Fix Committed
tags: added: verification-needed verification-needed-bionic
Revision history for this message
Dan Watkins (oddbloke) wrote :

I have built an image from -proposed and the .tar.xz was included in the output; verification done.

tags: added: verification-done verification-done-bionic
removed: verification-needed verification-needed-bionic
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package livecd-rootfs - 2.525.6

---------------
livecd-rootfs (2.525.6) bionic; urgency=medium

  [ Steve Langasek ]
  * generate all tar files with --xattrs. LP: #1302192.

  [ Daniel Watkins ]
  * ubuntu-cpc: Reintroduce the -root.tar.xz artifact (LP: #1585233).
  * ubuntu-cpc: Generate the root image contents once, and use it for both the
    -root.tar.xz and the .squashfs.
  * ubuntu-cpc: Generate -root.tar.xz with --xattrs.

 -- Steve Langasek <email address hidden> Mon, 06 Aug 2018 14:16:04 -0700

Changed in livecd-rootfs (Ubuntu Bionic):
status: Fix Committed → Fix Released
Revision history for this message
Łukasz Zemczak (sil2100) wrote : Update Released

The verification of the Stable Release Update for livecd-rootfs has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Łukasz Zemczak (sil2100) wrote :

Damn, I actually released it to bionic-updates without making sure: are the changes also released already in cosmic? Looking at it now, I didn't see a changelog entry for the -root.tar.xz in the cosmic livecd-rootfs.

Revision history for this message
Robert C Jennings (rcj) wrote : Re: [Bug 1585233] Re: Provide -root.tar.xz for bionic and later

On 08/20/2018 06:18 AM, Łukasz Zemczak wrote:
> Damn, I actually released it to bionic-updates without making sure: are
> the changes also released already in cosmic? Looking at it now, I didn't
> see a changelog entry for the -root.tar.xz in the cosmic livecd-rootfs.

Fear not, the changes are in trunk and the latest Cosmic dailies[1] ship
a -root.tar.xz.

[1] http://cloud-images.ubuntu.com/cosmic/current/

Revision history for this message
Server Team CI bot (server-team-bot) wrote :

This bug is fixed with commit c8838629 to simplestreams on branch master.
To view that commit see the following URL:
https://git.launchpad.net/simplestreams/commit/?id=c8838629

Revision history for this message
Dan Watkins (oddbloke) wrote :

This is now fixed.

Changed in cloud-images:
status: Confirmed → Fix Released
tags: added: id-5af30ba319eea4a3fee01c14
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Duplicates of this bug

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.