Merge lp:~laney/ubuntu-archive-tools/update-output-helper into lp:ubuntu-archive-tools

[Iain Lane]

This is a script which constructs a view of the archive that more or less corresponds to one that britney is trying to create. You can use that to analyse why britney is saying that packages aren't able to migrate.

[lp:~laney/ubuntu-archive-tools/update-output-helper]

973. By Colin Watson on 2015-08-18

[r=cjwatson] Add the team looking after ippusbxd in main.

974. By Colin Watson on 2015-08-24

manage-chroot: Log in anonymously for get/info.

975. By Colin Watson on 2015-08-24

[r=cjwatson] sru-report: list kernel family packages in the -proposed cleanup section

976. By Colin Watson on 2015-08-31

[r=cjwatson] Add the pkg-ime team which is subscribed to a couple of packages in main.

[Iain Lane (laney)]

I should fix this to make it a wrapper (or something similar) before we merge - the interface is a bit weird currently.

[lp:~laney/ubuntu-archive-tools/update-output-helper]

977. By Martin Pitt on 2015-09-22

Add retry-autopkgtest-regressions

This generates a set of run-autopkgtest commands (from britney2-ubuntu branch)
to re-run all autopkgtests which regressed.

978. By Martin Pitt on 2015-10-09

sru-report: Update YAML parsing for autopkgtest regressions

979. By Colin Watson on 2015-10-20

copy-package: Fix same-destination check: identical archives compare unequal (thanks, launchpadlib), so compare archive references.

980. By Colin Watson on 2015-10-22

branch-livefses: Revert r956 and materialise the livefses collection up-front instead.

981. By Colin Watson on 2015-10-22

wily → xenial / vivid → wily

982. By Colin Watson on 2015-10-23

copy-package: Make success output less misleading.

983. By Colin Watson on 2015-12-08

Add s390x to various reports for xenial.

984. By Martin Pitt on 2015-12-15

retry-autopkgtest-regressions: Stop assuming number of info URLs in YAML excuses

985. By Colin Watson on 2016-01-06

copy-package: Pass source series/pocket to Archive.copyPackage.

986. By Martin Pitt on 2016-02-01

copy-package: Add sponsor option

987. By Steve Langasek on 2016-02-05

Merge lp:~brian-murray/ubuntu-archive-tools/link-to-autopkgtest-page

988. By Colin Watson on 2016-02-05

component-mismatches: remove unused code

989. By Colin Watson on 2016-02-05

bootstrap-package: new tool to bootstrap a single package using injected build-dependencies

990. By Steve Langasek on 2016-02-09

Merge lp:~brian-murray/ubuntu-archive-tools/unicode-traceback

991. By William Grant on 2016-02-09

manage-builders: Update column widths to better match modern data.

992. By William Grant on 2016-02-09

manage-builders: Increase Cleaning warning threshold from 5 to 10 minutes, as mass resets can be slow.

993. By William Grant on 2016-02-09

[r=wgrant] manage-builders: Add --set-vm-host option.

994. By Colin Watson on 2016-02-10

manage-builders: add --quiet option

995. By Martin Pitt on 2016-02-25

retry-autopkgtest-regressions: Add --all-proposed option when running under $ALL_PROPOSED

996. By Adam Conrad on 2016-02-28

copy-proposed-kernel: Use new kernel security PPA for --security

997. By Steve Langasek on 2016-02-29

Merge lp:~brian-murray/ubuntu-archive-tools/add-snappy-dev

998. By Colin Watson on 2016-03-03

manage-builders: add --set-virtual and --set-non-virtual options

999. By Colin Watson on 2016-03-03

bootstrap-package: allow bootstrapping builds that are still "Needs building"

1000. By Martin Pitt on 2016-03-04

retry-autopkgtest-regressions: Use argparse

1001. By Martin Pitt on 2016-03-04

retry-autopkgtest-regressions: Add --state option

This is useful to retry all RUNNING tests in case test queues got
flushed for maintenance.

1002. By Martin Pitt on 2016-03-04

retry-autopkgtest-regressions: Add --ci-train option

1003. By Martin Pitt on 2016-03-11

retry-autopkgtest-regressions: Add --max-age option

1004. By Martin Pitt on 2016-03-11

retry-autopkgtest-regressions: Add --min-age option

1005. By Steve Langasek on 2016-03-30

Merge lp:~brian-murray/ubuntu-archive-tools/mir-bug-creation-improvements

1006. By Steve Langasek on 2016-04-09

Explicitly spell out the bug statuses we look for when searching MIR bugs to include wontfix, so that previously-rejected MIRs are visible on the report.

1007. By Steve Langasek on 2016-04-09

Re-merge with trunk

1008. By Steve Langasek on 2016-04-12

Drop incomplete with/without response from bug status list, already encompassed by 'incomplete' - thanks, Brian\!

1009. By Adam Conrad on 2016-04-13

priority-mismatches: Remove s390-tools workarounds

1010. By Steve Langasek on 2016-04-14

Merge lp:~brian-murray/ubuntu-archive-tools/unsubbed-dev-release

1011. By Colin Watson on 2016-04-20

auto-sync, kernel-overrides, orphaned-sources: stop relying on {Packages,Sources}.bz2; use .gz for now because I'm lazy

1012. By Colin Watson on 2016-04-20

package-subscribers: remove unused import

1013. By Colin Watson on 2016-04-22

xenial → yakkety; wily → xenial

1014. By Steve Langasek on 2016-04-27

Recognize maas-maintainers as a main team subscriber

1015. By Martin Pitt on 2016-04-28

retry-autopkgtest-regressions: Fix crash on None age, add missing parenthesis

1016. By Steve Langasek on 2016-04-29

When opening bugs, use the right status - Incomplete (without response) - so that they'll be indexed on the package bug page; 'Incomplete' is a legacy status that shouldn't be used

1017. By Steve Langasek on 2016-05-03

Add ubuntu-openstack team to list of known main owners

1018. By Steve Langasek on 2016-05-11

Add 'cloudware' as an owning team for packages on the cloud images

1019. By Steve Langasek on 2016-05-11

remove cloudware again, this is a private team that we can't query

1020. By Steve Langasek on 2016-05-12

Add support for a -e option, to allow specifying the exact version of a
package to review in the queue (rather than forcing you to manually process
everything when more than one version is present).

1021. By Steve Langasek on 2016-05-12

Merge lp:~brian-murray/ubuntu-archive-tools/phasing-ignore-dpkg-errors

1022. By Steve Langasek on 2016-06-03

Merge lp:~brian-murray/ubuntu-archive-tools/sru-report-table-id

1023. By Iain Lane on 2016-06-03

merge

1024. By Iain Lane on 2016-06-03

xz

Unmerged revisions

1024. By Iain Lane on 2016-06-03

xz

1023. By Iain Lane on 2016-06-03

merge

=== added file 'update-output-helper'
--- update-output-helper	1970-01-01 00:00:00 +0000
+++ update-output-helper	2016-06-03 10:32:32 +0000
@@ -0,0 +1,128 @@
+#!/bin/bash
+
+# Run me like this:
+# Download all the Packages files
+# update-output-helper -u
+# Create a worldview that britney is trying to mutate us to
+# update-output-helper package/version package2/version2/ ...
+# outputs an apt-get commandline to use
+# Then call apt with this world view
+# <apt-get command> update
+# <apt-get command> --dry-run install <some uninstallable package>
+# and you can drill down as normal to find which package(s) need fixing
+
+set -e
+
+GETOPT=$(getopt --options uh --long update,help -n "${0}" -- "$@")
+
+eval set -- "${GETOPT}"
+
+ARCH=${ARCH:-$(dpkg-architecture -qDEB_HOST_ARCH)}
+CACHEDIR=${XDG_CACHE_HOME:-$HOME/.cache}/brapt
+APTDIR=${CACHEDIR}/aptroot
+RELEASE=$(ubuntu-distro-info --devel)
+RELEASE_PACKAGES="${CACHEDIR}/Packages_${RELEASE}"
+PROPOSED_PACKAGES="${CACHEDIR}/Packages_${RELEASE}-proposed"
+
+[ ! -d "${CACHEDIR}" ] && mkdir -p "${CACHEDIR}"
+[ ! -d "${APTDIR}" ] && mkdir -p "${APTDIR}"
+
+ensure_programs() {
+    if [ $# -gt 0 ] && ! type "${1}" >/dev/null 2>/dev/null; then
+        echo "Required program $1 not found."
+        exit 1
+    fi
+
+    shift
+
+    if [ $# -gt 0 ]; then
+        ensure_programs $*
+    fi
+}
+
+ensure_apt_structure() {
+    mkdir -p "${APTDIR}/etc/apt/preferences.d" \
+        "${APTDIR}/var/cache/apt/archives/partial" \
+        "${APTDIR}/var/lib/apt/lists"
+    [ ! -e "etc/apt/sources.list" ] && \
+        echo "deb file://${APTDIR} ./" > "${APTDIR}/etc/apt/sources.list"
+    touch "${APTDIR}/status"
+}
+
+update() {
+    case ${ARCH} in
+        i386|amd64)
+            local MIRROR="http://archive.ubuntu.com/ubuntu"
+            ;;
+        *)
+            local MIRROR="http://ports.ubuntu.com"
+            ;;
+    esac
+
+    local COMPONENTS=(main universe multiverse restricted)
+    for series in "${RELEASE}" "${RELEASE}-proposed"; do
+        local FILE="${CACHEDIR}/Packages_${series}"
+        [ -e "${FILE}" ] && rm "${FILE}"
+        for component in "${COMPONENTS[@]}"; do
+            local URL="${MIRROR}/dists/${series}/${component}/binary-${ARCH}/Packages.xz"
+            echo "Downloading ${URL}"
+            wget -q -O - "${URL}" | xzcat >> "${CACHEDIR}/Packages_${series}"
+        done
+    done
+}
+
+while true; do
+    case "$1" in
+        -u|--update)
+            update
+            shift
+            ;;
+        -h|--help)
+            echo "Usage: $0 [-u] package/version ..."
+            shift
+            ;;
+        --)
+            shift
+            break
+            ;;
+        *)
+            echo "Error"
+            exit 1;
+            ;;
+    esac
+done
+
+if [ $# -eq 0 ]; then
+    exit 0
+fi
+
+while [ $# -gt 0 ]; do
+    PKG=${1%/*}
+    VERSION=${1#*/}
+    shift
+
+    if [ "${PKG}" = "${VERSION}" ]; then
+        echo "Error: ${PKG} is not in the format PACKAGE/VERSION" >&2
+        exit 1
+    fi
+
+    if [ $# -gt 0 ]; then
+        O="-o "
+    else
+        O=
+    fi
+
+    PROPOSED_PATTERN="${PROPOSED_PATTERN:- }( -X -S ${PKG} -a -FVersion ${VERSION} ) ${O}"
+    RELEASE_PATTERN="${RELEASE_PATTERN:- }( -X -S ${PKG} -a -! -FVersion ${VERSION} ) ${O}"
+done
+
+ensure_programs wget grep-dctrl xzcat
+
+ensure_apt_structure
+
+cat ${RELEASE_PACKAGES[@]} | grep-dctrl -v ${RELEASE_PATTERN} > ${APTDIR}/Packages
+cat ${PROPOSED_PACKAGES[@]} | grep-dctrl ${PROPOSED_PATTERN} >> ${APTDIR}/Packages
+
+apt-get -oDir=${APTDIR} -oDir::State::status=${APTDIR}/status update
+
+echo "apt-get -oDir=${APTDIR} -oDir::State::status=${APTDIR}/status --dry-run install <some broken package> ..."