Merge lp:~costamagnagianfranco/ubuntu-archive-tools/sync into lp:ubuntu-archive-tools

Proposed by Gianfranco Costamagna on 2019-10-19
Status: Rejected
Rejected by: Steve Langasek on 2021-02-11
Proposed branch: lp:~costamagnagianfranco/ubuntu-archive-tools/sync
Merge into: lp:ubuntu-archive-tools
Diff against target: 40 lines (+7/-4)
1 file modified
auto-sync (+7/-4)
To merge this branch: bzr merge lp:~costamagnagianfranco/ubuntu-archive-tools/sync
Reviewer Review Type Date Requested Status
Steve Langasek 2019-10-19 Disapprove on 2021-02-11
Review via email: mp+374390@code.launchpad.net

Description of the change

example of log file:
$ ./auto-sync --dry-run --batch
Reading Ubuntu sources ...
Reading Debian sources ...
Getting differences between Debian/sid and Ubuntu/eoan-proposed ...
[Updating] binutils-mipsen (2~c2 [Ubuntu] < 2~c3 [Debian])
 * Trying to add binutils-mipsen ...
binutils-mipsen_2~c3 is trying to override modified binary binutils-mips64-linux-gnuabi64_2.32.51.20190702-1ubuntu1. OK (y/N)? n
Previous publication for dateutils with version: 0.4.5-1 found in Ubuntu. Not syncing version: 0.4.5-1 from Debian
[Updating] dictem (1.0.4-2 [Ubuntu] < 1.0.4-4 [Debian])
 * Trying to add dictem ...
I: dictem -> dictem_1.0.4-2.
Previous publication for hidrd with version: 0.2.0-12 found in Ubuntu. Not syncing version: 0.2.0-12 from Debian
Previous publication for osmo-hlr with version: 0.2.1-4 found in Ubuntu. Not syncing version: 0.2.1-4 from Debian
Previous publication for pgpdump with version: 0.33-1ubuntu1 found in Ubuntu. Not syncing version: 0.33-1 from Debian
Previous publication for php-horde-icalendar with version: 2.1.8-1 found in Ubuntu. Not syncing version: 2.1.8-1 from Debian
Previous publication for php-horde-image with version: 2.5.3-1 found in Ubuntu. Not syncing version: 2.5.3-1 from Debian
[Updating] quotatool (1:1.4.12-2build1 [Ubuntu] < 1:1.6.2-5 [Debian])
 * Trying to add quotatool ...
I: quotatool -> quotatool_1:1.4.12-2build1.
Previous publication for stgit with version: 0.18-1 found in Ubuntu. Not syncing version: 0.18-1 from Debian
[Updating] sysvbanner (1.0.15build1 [Ubuntu] < 1:1.0-16 [Debian])
 * Trying to add sysvbanner ...
I: sysvbanner -> sysvbanner_1.0.15build1.
Previous publication for v-sim with version: 3.7.2-6 found in Ubuntu. Not syncing version: 3.7.2-6 from Debian
Previous publication for vnstat with version: 1.18-2 found in Ubuntu. Not syncing version: 1.18-2 from Debian
[New] ayatana-indicator-messages_0.6.0-2
No previous publications in Ubuntu
OK (Y/n)? y
 * Trying to add ayatana-indicator-messages ...
ayatana-indicator-messages_0.6.0-2 is trying to override modified binary libmessaging-menu0_13.10.1+18.10.20180918-0ubuntu1. OK (y/N)? n
[New] baloo-widgets_4:18.08.1-1
Previous publications in Ubuntu:
 4:15.12.3-0ubuntu1 (yakkety): Deleted (removed by Steve Langasek: (From Debian) ROM; dead upstream, no more used; Debian bug #829290)
 4:15.12.3-0ubuntu1 (xenial): Published
 4:15.12.3-0ubuntu1 (xenial): Deleted (removed by Ubuntu Archive Robot: moved to release)
 4:15.12.1-0ubuntu1 (xenial): Superseded
 4:15.12.1-0ubuntu1 (xenial): Deleted (removed by Ubuntu Archive Robot: moved to release)
 4:15.08.2-0ubuntu1 (xenial): Superseded
 4:15.08.2-0ubuntu1 (wily): Obsolete
 4:15.08.2-0ubuntu1 (wily): Deleted (removed by Ubuntu Archive Robot: moved to release)
 4:15.08.1-0ubuntu1 (wily): Superseded
 4:15.08.1-0ubuntu1 (wily): Deleted (removed by Ubuntu Archive Robot: moved to release)
 ... plus 30 more; see https://launchpad.net/ubuntu/+source/baloo-widgets/+publishinghistory
OK (Y/n)? y
 * Trying to add baloo-widgets ...
baloo-widgets_4:18.08.1-1 is trying to override modified binary libkf5baloowidgets-bin_4:19.04.2-0ubuntu1. OK (y/N)? n
[New] compiz-plugins-extra_2:0.8.16-2
Previous publications in Ubuntu:
 0.9.7.0~bzr9-0ubuntu7 (quantal): Deleted (removed by Colin Watson: requested by mitya57; obsolete source package; LP: #1028363)
 0.9.7.0~bzr9-0ubuntu6 (quantal): Superseded
 0.9.7.0~bzr9-0ubuntu6 (precise): Published
 0.9.7.0~bzr9-0ubuntu5 (precise): Superseded
 0.9.7.0~bzr9-0ubuntu4 (precise): Superseded
 0.9.7.0~bzr9-0ubuntu4 (precise): Deleted (removed by Colin Watson: moved to release)
 0.9.7.0~bzr9-0ubuntu3 (precise): Superseded
 0.9.7.0~bzr9-0ubuntu2 (precise): Superseded
 0.9.7.0~bzr9-0ubuntu1 (precise): Superseded
 0.9.5.94-0ubuntu1 (precise): Superseded
 ... plus 7 more; see https://launchpad.net/ubuntu/+source/compiz-plugins-extra/+publishinghistory
OK (Y/n)? y
 * Trying to add compiz-plugins-extra ...
compiz-plugins-extra_2:0.8.16-2 is trying to override modified binary compiz-plugins-extra_1:0.9.14.0+19.04.20190223.1-0ubuntu1. OK (y/N)? n
[New] compiz-plugins-main_2:0.8.16-4
Previous publications in Ubuntu:
 1:0.9.7.0~bzr19-0ubuntu11 (quantal): Deleted (removed by Adam Conrad: Superseded by compiz)
 1:0.9.7.0~bzr19-0ubuntu10.1 (precise): Deleted (removed by Colin Watson: moved to -updates)
 1:0.9.7.0~bzr19-0ubuntu10 (quantal): Superseded
 1:0.9.7.0~bzr19-0ubuntu10 (precise): Published
 1:0.9.7.0~bzr19-0ubuntu9 (precise): Superseded
 1:0.9.7.0~bzr19-0ubuntu8 (precise): Superseded
 1:0.9.7.0~bzr19-0ubuntu7 (precise): Superseded
 1:0.9.7.0~bzr19-0ubuntu7 (precise): Deleted (removed by Colin Watson: moved to release)
 1:0.9.7.0~bzr19-0ubuntu6 (precise): Superseded
 1:0.9.7.0~bzr19-0ubuntu5 (precise): Superseded
 ... plus 31 more; see https://launchpad.net/ubuntu/+source/compiz-plugins-main/+publishinghistory
OK (Y/n)? y
 * Trying to add compiz-plugins-main ...
compiz-plugins-main_2:0.8.16-4 is trying to override modified binary compiz-plugins-main_1:0.9.14.0+19.04.20190223.1-0ubuntu1. OK (y/N)? n
[New] gcc-8-doc_8.3.0-1
No previous publications in Ubuntu
OK (Y/n)? y
 * Trying to add gcc-8-doc ...
gcc-8-doc_8.3.0-1 is trying to override modified binary gcc-8-doc_8.3.0-19ubuntu1. OK (y/N)? n
[New] google-compute-image-packages_20190124-3
No previous publications in Ubuntu
OK (Y/n)? y
 * Trying to add google-compute-image-packages ...
google-compute-image-packages_20190124-3 is trying to override modified binary python3-google-compute-engine_20190522-0ubuntu2. OK (y/N)? n
[New] libcompizconfig_2:0.8.16-2
Previous publications in Ubuntu:
 0.9.7.0~bzr428-0ubuntu7 (quantal): Deleted (removed by Adam Conrad: Superseded by compiz)
 0.9.7.0~bzr428-0ubuntu6 (quantal): Superseded
 0.9.7.0~bzr428-0ubuntu6 (precise): Published
 0.9.7.0~bzr428-0ubuntu6 (precise): Deleted (removed by Colin Watson: moved to release)
 0.9.7.0~bzr428-0ubuntu4 (precise): Superseded
 0.9.7.0~bzr428-0ubuntu3 (precise): Superseded
 0.9.7.0~bzr428-0ubuntu2 (precise): Superseded
 0.9.7.0~bzr428-0ubuntu1 (precise): Superseded
 0.9.5.94-0ubuntu2 (precise): Superseded
 0.9.5.94-0ubuntu2 (oneiric): Obsolete
 ... plus 72 more; see https://launchpad.net/ubuntu/+source/libcompizconfig/+publishinghistory
OK (Y/n)? y
 * Trying to add libcompizconfig ...
libcompizconfig_2:0.8.16-2 is trying to override modified binary libcompizconfig0-dev_1:0.9.14.0+19.04.20190223.1-0ubuntu1. OK (y/N)? n

Out-of-date BUT modified: 379 (52.28%)
Updated: 3 (0.41%)
Ubuntu Specific: 0 (0.00%)
Up-to-date [Modified]: 0 (0.00%)
Up-to-date: 0 (0.00%)
Blacklisted: 111 (15.31%)
                          -----------
Total: 725

Updating: dictem quotatool sysvbanner
Not copying packages in dry-run mode.

To post a comment you must log in.
1247. By Colin Watson on 2019-10-21

component-mismatches: fix sorting of publications to not assume comparability of Entry instances in py3

1248. By Steve Langasek on 2019-10-29

call reverse-depends with -a source instead of with -b, in order to show both reverse-build-depends and reverse-test-depends (ubuntu-dev-tools 0.174 and later)

1249. By Adam Conrad on 2019-10-31

sru-report: Ignore more linux- packages.

1250. By Łukasz Zemczak on 2019-11-04

Initial support for kernel routing lookups via KernelSeries data from the kernel team.

1251. By Andy Whitcroft on 2019-11-05

kernel_series: resync with upstream

Resync with lp:~canonical-kernel-team/+git/kteam-tools ktl/kernel_series.py.

Signed-off-by: Andy Whitcroft <email address hidden>

1252. By Steve Langasek on 2019-12-05

Merge lp:~vorlon/ubuntu-archive-tools/update-i386-whitelist

1253. By Steve Langasek on 2019-12-05

Enforce interactive confirmation when changing the i386 whitelist

1254. By Steve Langasek on 2019-12-05

Correct syntax for adding to a set

1255. By Steve Langasek on 2019-12-08

hard-code some new dependencies of lintian

1256. By Łukasz Zemczak on 2019-12-16

Add the new kernels to the exclusion list for sru-report.

1257. By Matthias Klose on 2019-12-16

 update-i386-whitelist: add gcc-10-cross and gcc-10-cross-ports

1258. By Steve Langasek on 2019-12-19

Merge lp:~xnox/ubuntu-archive-tools/unbreak-nusakan

1259. By Steve Langasek on 2019-12-23

manually exclude frei0r for now

1260. By Steve Langasek on 2020-01-08

handle the case that frei0r is already absent from the packageset (because binaries are gone from the archive, as they are now); and add spirv-tools to the bootstrap list for new glslang in -proposed.

1261. By Steve Langasek on 2020-01-10

don't include xorg in the whitelist, only the arch: all packages are used

1262. By Steve Langasek on 2020-01-16

Add ubuntu-drivers-common to the packages manually excluded from the seed

1263. By Steve Langasek on 2020-01-22

add linux-5.4 to the set, as the new source for linux-libc-dev :/

1264. By Steve Langasek on 2020-01-22

bootstrap ruby transition

1265. By Matthias Klose on 2020-01-25

change shebangs from python to python2.7

1266. By Steve Langasek on 2020-01-28

manual overrides for new -proposed build-dependencies

1267. By Steve Langasek on 2020-01-31

bootstrap boost1.71

1268. By Steve Langasek on 2020-01-31

bootstrap of new hwloc finished

1269. By Steve Langasek on 2020-02-03

linux-5.4 no longer needs special-cased

1270. By Steve Langasek on 2020-02-07

gspell bootstrap done

1271. By Steve Langasek on 2020-02-07

Merge lp:~raof/ubuntu-archive-tools/python3

1272. By Steve Langasek on 2020-02-07

gcc-10 is in-archive

1273. By Steve Langasek on 2020-02-07

bootstrap of new lintian done.

1274. By Steve Langasek on 2020-02-07

erge lp:~dannf/ubuntu-archive-tools/py3-copy-package

1275. By Colin Watson on 2020-02-19

manage-builders: add --idle filter option

1276. By Colin Watson on 2020-02-19

manage-builders: run with python3

1277. By Iain Lane on 2020-02-24

oem-metapackage-mir-check: Add script to check oem-*-meta MIR exceptions

This was requested by the MIR team. Checks the given source package against the
reference package in the Ubuntu archive (oem-qemu-meta) so that archive admins
can check for compliance with the MIR exception
(https://wiki.ubuntu.com/MIRTeam/Exceptions/OEM) and promote or NEW directly
into main.

1278. By Steve Langasek on 2020-02-25

boost1.71 bootstrap done; php7.4 needs bootstrapped now

1279. By Łukasz Zemczak on 2020-02-26

Print a warning about whenever the block proposed tag is present on any of the bugs via sru-review.

1280. By Steve Langasek on 2020-02-26

bootstrap libfido2/openssh

1281. By Steve Langasek on 2020-02-26

bootstrap tepl/gedit

1282. By Steve Langasek on 2020-02-26

bootstrap fmtlib/spdlog

1283. By Steve Langasek on 2020-02-27

include libcbor for openssh

1284. By William Grant on 2020-02-27

Port queue to Python 3.

1285. By Sebastien Bacher on 2020-03-02

Switch some of the script to use python3 so they work on focal

1286. By Steve Langasek on 2020-03-02

drop finished bootstraps

1287. By Steve Langasek on 2020-03-02

bootstrap libxcrypt

1288. By Łukasz Zemczak on 2020-03-06

Try different wording for the SRU acceptance message, adding words for including verification steps.

1289. By Łukasz Zemczak on 2020-03-09

Add new kernels to the SRU report exclusion list.

1290. By Steve Langasek on 2020-03-09

bootstrap llvm 10

1291. By Steve Langasek on 2020-03-11

bootstrap new webkit2gtk

1292. By Steve Langasek on 2020-03-11

Merge lp:~laney/ubuntu-archive-tools/mozjs68

1293. By Steve Langasek on 2020-03-12

libwpe also needed for webkit2gtk

1294. By Steve Langasek on 2020-03-16

bootstrap new bind9, lintian deps

1295. By Steve Langasek on 2020-03-24

prune whitelist for finished transitions

1296. By Colin Watson on 2020-03-27

manage-chroot: fix rollback output to include "set"

1297. By Colin Watson on 2020-03-27

manage-chroot: fix parsing of LP URLs (no mangling for those already on service host)

1298. By Steve Langasek on 2020-03-31

drop special-casing of finished bootstraps

1299. By Steve Langasek on 2020-03-31

drop special-casing of finished bootstraps

1300. By Steve Langasek on 2020-03-31

drop special-casing of finished bootstraps

1301. By Łukasz Zemczak on 2020-04-02

Add armhf+raspi and arm64+raspi to publish-image-set

1302. By Łukasz Zemczak on 2020-04-03

Ok, raspi is preinstalled as well.

1303. By Colin Watson on 2020-04-04

new-binary-debian-universe: simplify urllib imports using six

1304. By Colin Watson on 2020-04-04

new-binary-debian-universe: port to Python 3

1305. By Colin Watson on 2020-04-06

manage-chroot: port to Python 3

1306. By Steve Langasek on 2020-04-09

drop override for finished glslang bootstrap

1307. By Matthias Klose on 2020-04-15

update-i386-whitelist: add openjdk-12, required to bootstrap openjdk-13

1308. By Matthias Klose on 2020-04-15

update-i386-whitelist: add openjdk-N

1309. By Łukasz Zemczak on 2020-04-16

Add legacy-server

1310. By Steve Langasek on 2020-04-16

include riscv64 in c-m archive list

1311. By Matthias Klose on 2020-04-20

Correctly add riscv64 to components-mismatches (xnox)

1312. By Matthias Klose on 2020-04-24

focal -> groovy for devel; eoan -> focal for SRU

1313. By Colin Watson on 2020-04-24

mark-suite-dirty: port to Python 3

1314. By Łukasz Zemczak on 2020-04-24

Make branch-seeds wait a bit for the new git refs to appear before attempting to set the default_branch

1315. By Steve Langasek on 2020-04-24

branch-livefses: move to python3

1316. By Steve Langasek on 2020-04-24

branch-livefses: if the livefs already exists, just use it

1317. By Steve Langasek on 2020-04-24

branch-livefses: use specific lazr exceptions instead of 'Exception'; and be more useful when we don't have permissions to devirt a ppa.

1318. By Colin Watson on 2020-04-24

branch-livefses: convert to argparse

1319. By Colin Watson on 2020-04-24

branch-livefses: invert new vs. existing exception handling

This way round works better for the case where the user has permissions to
set require_virtualized and relative_build_score but not to actually create
the livefs (assuming somebody else has already done that), and should still
be fine, if slower, for the converse set of permissions.

1320. By Colin Watson on 2020-04-27

edit-acl: convert to argparse

1321. By Colin Watson on 2020-04-27

edit-acl: port to Python 3

1322. By Łukasz Zemczak on 2020-05-13

Switch copy-proposed-kernel to python3.

1323. By Steve Langasek on 2020-05-15

move sru-report to python3

1324. By Steve Langasek on 2020-05-16

sru-report: handle utf-8 output on stdout

1325. By Steve Langasek on 2020-05-20

Merge lp:~brian-murray/ubuntu-archive-tools/py3-sru-accept

1326. By Łukasz Zemczak on 2020-05-22

sru-remove can be switched to python 3 as well.

1327. By Steve Langasek on 2020-05-28

Merge lp:~xnox/ubuntu-archive-tools/built-using-finally

1328. By Steve Langasek on 2020-05-28

don't pass architecture=arch to apt_pkg.parse_depends, not supported on xenial.

1329. By Steve Langasek on 2020-05-28

Merge lp:~brian-murray/ubuntu-archive-tools/pu-stopped-email

1330. By Steve Langasek on 2020-05-28

use https not http for yui

1331. By Łukasz Zemczak on 2020-05-28

kernel-sru-review: look at kernel-stable-master-bug for the master bug in the description first, and only then fallback to the tags. The tags can sometimes be out-of-date.

1332. By Iain Lane on 2020-06-09

Resurrect libemail-address-xs-perl, new lintian dep

1333. By Sebastien Bacher on 2020-06-10

revert temporary change from the previous commit

1334. By Steve Langasek on 2020-06-17

Merge lp:~vorlon/ubuntu-archive-tools/clean-NBS-for-EOL

1335. By Steve Langasek on 2020-06-17

Merge lp:~brian-murray/ubuntu-archive-tools/identify-security

1336. By Steve Langasek on 2020-06-17

s/focal/groovy

1337. By Steve Langasek on 2020-06-17

Drop override for finished spdlog boostrap

1338. By Steve Langasek on 2020-06-22

Allow canonical-support as an owning team for packages in main

1339. By Steve Langasek on 2020-06-30

Fix retry-autopkgtest-regressions for compatibility with the new britney yaml schema

1340. By Steve Langasek on 2020-07-01

Fix sru-report for compatibility wit the new britney yaml schema

1341. By Steve Langasek on 2020-07-01

Merge lp:~laney/ubuntu-archive-tools/r-a-r-excuses-xz

1342. By Steve Langasek on 2020-07-02

Merge lp:~vorlon/ubuntu-archive-tools/close-EOL-bugs

1343. By Gianfranco Costamagna on 2020-07-03

modify auto-sync script to not sync if the same version has been published also in a previous Ubuntu version, not only the current devel one

Steve Langasek (vorlon) wrote :

Sorry for taking so long to review this; I looked at it briefly when it was first raised but it took some time to think through the subtleties.

I think we should NOT merge this, for a subtle reason. We currently use the extra-removals.txt in the sync-blacklist repository to document packages whose sources have been removed because they *depend on* packages that have been removed for bugginess, rather than because they are buggy themselves.

Without this change, those packages will be re-synced automatically during the next release. With this change, they will not.

Given the tradeoff between missing some of these packages which have become un-buggy and failing to resync them in the next series, and archive admins having to re-remove them in the next series if they are still broken, I think it is better to do the extra work to re-remove them, because it is worse to miss a non-buggy package in a release (or to have its re-syncing delayed, possibly causing knock-on effects) than it is to miss removing a still-unreleasable package from -proposed.

I looked at this MP briefly when it was first raised. I'm generally ok with it, but the rationale is very subtle and I think it needs to be called out here, either in a code comment or at minimum in the commit message.

Specifically, the reason it is correct to not re-sync a source package version that was previously synced and then removed is that we have an unwritten policy of only removing source packages from -proposed if that version of that package is buggy in both Debian and Ubuntu and requires sourceful changes to that package in order to be

review: Disapprove

Unmerged revisions

1343. By Gianfranco Costamagna on 2020-07-03

modify auto-sync script to not sync if the same version has been published also in a previous Ubuntu version, not only the current devel one

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'auto-sync'
2--- auto-sync 2017-12-13 19:26:54 +0000
3+++ auto-sync 2020-07-03 12:25:25 +0000
4@@ -337,21 +337,24 @@
5 return True
6
7
8-def already_in_target_series(options, difference):
9+def already_in_all_target_archive(options, difference):
10 # The published Sources files may be out of date, and if we're
11 # particularly unlucky with timing relative to a proposed-migration run
12 # it's possible for them to miss something that's in the process of
13 # being moved between pockets. To make sure, check whether an equal or
14- # higher version has already been removed from the destination archive.
15+ # higher version has already been removed from the destination archive
16+ # in all possible target series.
17 src = difference.sourcepackagename
18 from_version = difference.parent_source_version
19 to_src = version_sort_spphs(filter_pockets(
20 options.target.archive.getPublishedSources(
21- distro_series=options.target.series, source_name=src,
22+ source_name=src,
23 exact_match=True)))
24 if (to_src and
25 apt_pkg.version_compare(
26 from_version, to_src[0].source_package_version) <= 0):
27+ print("Previous publication for " + src + " with version: " + to_src[0].source_package_version
28+ + " found in Ubuntu. Not syncing version: " + from_version + " from Debian")
29 return True
30 else:
31 return False
32@@ -571,7 +574,7 @@
33 else:
34 if not published_in_source_series(options, difference):
35 continue
36- if already_in_target_series(options, difference):
37+ if already_in_all_target_archive(options, difference):
38 continue
39 print("[Updating] %s (%s [%s] < %s [%s])" % (
40 src, to_version,

Subscribers

People subscribed via source and target branches