ubuntu-cpc parallel builds produce unused files

Bug #1837254 reported by Robert C Jennings
8
This bug affects 1 person
Affects Status Importance Assigned to Milestone
livecd-rootfs (Ubuntu)
Fix Released
Undecided
Unassigned
Xenial
Fix Released
Undecided
Unassigned
Bionic
Fix Released
Undecided
Unassigned
Disco
Fix Released
Undecided
Unassigned

Bug Description

[Impact]

In parallel builds where a list of image targets are provided, the build
may produce binaries that are not part of the named set of targets but
are created by series dependencies. These implicitly created binaries
may be generated by multiple builds but are unused as our convention for
the ubuntu-cpc project is to only consume binaries from the explicitly
named image targets; this avoids overwriting the same object by multiple
parallel builds. The problem is that as we build out parallel jobs for
a number of image targets (multiplied by each supported architecture)
we're wasting a lot of resources.

[Test Case]

 * Specify an image target within the ubuntu-cpc project for a build

 * Observe artifacts in addition to the ones for the named image target are produced and downloaded

[Regression Potential]

 * An artifact from an explicitly named artifact is missing and builds will begin to remove desired output. Testing is being done to address this regression.

[Other Info]

 * This has been running in production in Eoan for a few weeks and we're satisfied with the results. We have seen overall time reduced by 20-50% on some builds.

[Original Description]

In parallel builds where a list of image targets are provided, the build
may produce binaries that are not part of the named set of targets but
are created by series dependencies. These implicitly created binaries
may be generated by multiple builds but are unused as our convention for
the ubuntu-cpc project is to only consume binaries from the explicitly
named image targets; this avoids overwriting the same object by multiple
parallel builds. The problem is that as we build out parallel jobs for
a number of image targets (multiplied by each supported architecture)
we're wasting a lot of resources.

Related branches

Revision history for this message
Robert C Jennings (rcj) wrote :

I have attached an MP. In testing with the ubuntu-cpc project on eoan, when building the amd64 qcow2 image target we eliminate 1.38GiB in unnecessary transfer and storage. This saves a lot of time, storage, and transit for each parallel build on each architecture (so it adds up fast).

Robert C Jennings (rcj)
description: updated
description: updated
tags: added: id-5d31721934a49770eea1ff43
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package livecd-rootfs - 2.603

---------------
livecd-rootfs (2.603) eoan; urgency=medium

  [ Tobias Koch ]
  * Add retry logic to snap-tool to make downloads more resilient

  [ Robert C Jennings ]
  * ubuntu-cpc: Only produce explicitly specified artifacts (LP: #1837254)

 -- Tobias Koch <email address hidden> Mon, 15 Jul 2019 17:59:06 +0200

Changed in livecd-rootfs (Ubuntu):
status: New → Fix Released
Robert C Jennings (rcj)
description: updated
description: updated
Revision history for this message
Robert C Jennings (rcj) wrote :

I've completed image build testing across xenial, bionic, and disco for the SRU MPs and they're ready for review and upload.

Revision history for this message
Łukasz Zemczak (sil2100) wrote :

I have reviewed and merged all the D/B/X branches for livecd-rootfs. Would you want me to sponsor those?

Revision history for this message
Robert C Jennings (rcj) wrote :

Yes, please. I would like them sponsored. Thank you!

Revision history for this message
Timo Aaltonen (tjaalton) wrote : Please test proposed package

Hello Robert, or anyone else affected,

Accepted livecd-rootfs into disco-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd-rootfs/2.578.8 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-disco to verification-done-disco. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-disco. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in livecd-rootfs (Ubuntu Disco):
status: New → Fix Committed
tags: added: verification-needed verification-needed-disco
Changed in livecd-rootfs (Ubuntu Bionic):
status: New → Fix Committed
tags: added: verification-needed-bionic
Revision history for this message
Timo Aaltonen (tjaalton) wrote :

Hello Robert, or anyone else affected,

Accepted livecd-rootfs into bionic-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd-rootfs/2.525.30 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-bionic to verification-done-bionic. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-bionic. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Changed in livecd-rootfs (Ubuntu Xenial):
status: New → Fix Committed
tags: added: verification-needed-xenial
Revision history for this message
Timo Aaltonen (tjaalton) wrote :

Hello Robert, or anyone else affected,

Accepted livecd-rootfs into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd-rootfs/2.408.51 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (livecd-rootfs/2.525.30)

All autopkgtests for the newly accepted livecd-rootfs (2.525.30) for bionic have finished running.
The following regressions have been reported in tests triggered by the package:

ubuntu-image/1.7+18.04ubuntu1 (arm64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/bionic/update_excuses.html#livecd-rootfs

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Robert C Jennings (rcj) wrote :

The autopkgtest failure was a transient network failure during the build. After log analysis the test was re-run and is passing. I will perform verification tests and update the bug with results in the next few days.

Revision history for this message
Adam Conrad (adconrad) wrote : Please test proposed package

Hello Robert, or anyone else affected,

Accepted livecd-rootfs into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd-rootfs/2.408.52 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (livecd-rootfs/2.408.52)

All autopkgtests for the newly accepted livecd-rootfs (2.408.52) for xenial have finished running.
The following regressions have been reported in tests triggered by the package:

ubuntu-image/1.7+16.04ubuntu1 (amd64)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/xenial/update_excuses.html#livecd-rootfs

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Adam Conrad (adconrad) wrote : Please test proposed package

Hello Robert, or anyone else affected,

Accepted livecd-rootfs into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd-rootfs/2.408.53 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Ubuntu SRU Bot (ubuntu-sru-bot) wrote : Autopkgtest regression report (livecd-rootfs/2.408.53)

All autopkgtests for the newly accepted livecd-rootfs (2.408.53) for xenial have finished running.
The following regressions have been reported in tests triggered by the package:

ubuntu-image/1.7+16.04ubuntu1 (i386)

Please visit the excuses page listed below and investigate the failures, proceeding afterwards as per the StableReleaseUpdates policy regarding autopkgtest regressions [1].

https://people.canonical.com/~ubuntu-archive/proposed-migration/xenial/update_excuses.html#livecd-rootfs

[1] https://wiki.ubuntu.com/StableReleaseUpdates#Autopkgtest_Regressions

Thank you!

Revision history for this message
Adam Conrad (adconrad) wrote : Please test proposed package

Hello Robert, or anyone else affected,

Accepted livecd-rootfs into xenial-proposed. The package will build now and be available at https://launchpad.net/ubuntu/+source/livecd-rootfs/2.408.54 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested and change the tag from verification-needed-xenial to verification-done-xenial. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed-xenial. In either case, without details of your testing we will not be able to proceed.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance for helping!

N.B. The updated package will be released to -updates after the bug(s) fixed by this package have been verified and the package has been in -proposed for a minimum of 7 days.

Revision history for this message
Robert C Jennings (rcj) wrote :

Test builds completed for ubuntu-cpc for xenial, bionic, and disco and I'm marking this verification done and ready for promotion.

tags: added: verification-done verification-done-bionic verification-done-disco verification-done-xenial
removed: verification-needed verification-needed-bionic verification-needed-disco verification-needed-xenial
Revision history for this message
Brian Murray (brian-murray) wrote : Update Released

The verification of the Stable Release Update for livecd-rootfs has completed successfully and the package is now being released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regressions.

Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package livecd-rootfs - 2.408.54

---------------
livecd-rootfs (2.408.54) xenial; urgency=medium

  * Revert exclusion of makedev from buildd chroots, as it turned out not to
    be the problem. Instead, fix up /dev/ptmx to be a character device node
    rather than a symlink to /dev/pts/ptmx, in line with the discussion in
    https://bugs.debian.org/817236; I think this is safer than
    cherry-picking the fix to debootstrap at this point in a stable release
    cycle (LP: #1844504).

livecd-rootfs (2.408.53) xenial; urgency=medium

  * Fix exclusion of makedev from buildd chroots; debootstrap doesn't
    respect --exclude for "Priority: required" packages, so we have to purge
    makedev later (LP: #1844504).

livecd-rootfs (2.408.52) xenial; urgency=medium

  * Exclude makedev from buildd chroots, since it was historically excluded
    and apparently breaks some builds (e.g. mir; LP: #1844504).

livecd-rootfs (2.408.51) xenial; urgency=medium

  * ubuntu-cpc: Only produce explicitly specified artifacts (LP: #1837254)

 -- Colin Watson <email address hidden> Thu, 26 Sep 2019 10:53:23 +0100

Changed in livecd-rootfs (Ubuntu Xenial):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package livecd-rootfs - 2.578.8

---------------
livecd-rootfs (2.578.8) disco; urgency=medium

  * ubuntu-cpc: Only produce explicitly specified artifacts (LP: #1837254)

 -- Robert C Jennings <email address hidden> Mon, 26 Aug 2019 16:25:24 -0500

Changed in livecd-rootfs (Ubuntu Disco):
status: Fix Committed → Fix Released
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package livecd-rootfs - 2.525.30

---------------
livecd-rootfs (2.525.30) bionic; urgency=medium

  [ Robert C Jennings ]
  * ubuntu-cpc: Only produce explicitly specified artifacts (LP: #1837254)

  [ Ethan Hsieh ]
  * Add support for the Nitrogen6x board (LP: #1838064).

 -- Łukasz 'sil2100' Zemczak <email address hidden> Tue, 10 Sep 2019 16:56:26 +0200

Changed in livecd-rootfs (Ubuntu Bionic):
status: Fix Committed → Fix Released
To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.