Merge lp:~costamagnagianfranco/ubuntu-archive-tools/sync into lp:ubuntu-archive-tools

Proposed by Gianfranco Costamagna
Status: Rejected
Rejected by: Steve Langasek
Proposed branch: lp:~costamagnagianfranco/ubuntu-archive-tools/sync
Merge into: lp:ubuntu-archive-tools
Diff against target: 40 lines (+7/-4)
1 file modified
auto-sync (+7/-4)
To merge this branch: bzr merge lp:~costamagnagianfranco/ubuntu-archive-tools/sync
Reviewer Review Type Date Requested Status
Steve Langasek Disapprove
Review via email: mp+374390@code.launchpad.net

Description of the change

example of log file:
$ ./auto-sync --dry-run --batch
Reading Ubuntu sources ...
Reading Debian sources ...
Getting differences between Debian/sid and Ubuntu/eoan-proposed ...
[Updating] binutils-mipsen (2~c2 [Ubuntu] < 2~c3 [Debian])
 * Trying to add binutils-mipsen ...
binutils-mipsen_2~c3 is trying to override modified binary binutils-mips64-linux-gnuabi64_2.32.51.20190702-1ubuntu1. OK (y/N)? n
Previous publication for dateutils with version: 0.4.5-1 found in Ubuntu. Not syncing version: 0.4.5-1 from Debian
[Updating] dictem (1.0.4-2 [Ubuntu] < 1.0.4-4 [Debian])
 * Trying to add dictem ...
I: dictem -> dictem_1.0.4-2.
Previous publication for hidrd with version: 0.2.0-12 found in Ubuntu. Not syncing version: 0.2.0-12 from Debian
Previous publication for osmo-hlr with version: 0.2.1-4 found in Ubuntu. Not syncing version: 0.2.1-4 from Debian
Previous publication for pgpdump with version: 0.33-1ubuntu1 found in Ubuntu. Not syncing version: 0.33-1 from Debian
Previous publication for php-horde-icalendar with version: 2.1.8-1 found in Ubuntu. Not syncing version: 2.1.8-1 from Debian
Previous publication for php-horde-image with version: 2.5.3-1 found in Ubuntu. Not syncing version: 2.5.3-1 from Debian
[Updating] quotatool (1:1.4.12-2build1 [Ubuntu] < 1:1.6.2-5 [Debian])
 * Trying to add quotatool ...
I: quotatool -> quotatool_1:1.4.12-2build1.
Previous publication for stgit with version: 0.18-1 found in Ubuntu. Not syncing version: 0.18-1 from Debian
[Updating] sysvbanner (1.0.15build1 [Ubuntu] < 1:1.0-16 [Debian])
 * Trying to add sysvbanner ...
I: sysvbanner -> sysvbanner_1.0.15build1.
Previous publication for v-sim with version: 3.7.2-6 found in Ubuntu. Not syncing version: 3.7.2-6 from Debian
Previous publication for vnstat with version: 1.18-2 found in Ubuntu. Not syncing version: 1.18-2 from Debian
[New] ayatana-indicator-messages_0.6.0-2
No previous publications in Ubuntu
OK (Y/n)? y
 * Trying to add ayatana-indicator-messages ...
ayatana-indicator-messages_0.6.0-2 is trying to override modified binary libmessaging-menu0_13.10.1+18.10.20180918-0ubuntu1. OK (y/N)? n
[New] baloo-widgets_4:18.08.1-1
Previous publications in Ubuntu:
 4:15.12.3-0ubuntu1 (yakkety): Deleted (removed by Steve Langasek: (From Debian) ROM; dead upstream, no more used; Debian bug #829290)
 4:15.12.3-0ubuntu1 (xenial): Published
 4:15.12.3-0ubuntu1 (xenial): Deleted (removed by Ubuntu Archive Robot: moved to release)
 4:15.12.1-0ubuntu1 (xenial): Superseded
 4:15.12.1-0ubuntu1 (xenial): Deleted (removed by Ubuntu Archive Robot: moved to release)
 4:15.08.2-0ubuntu1 (xenial): Superseded
 4:15.08.2-0ubuntu1 (wily): Obsolete
 4:15.08.2-0ubuntu1 (wily): Deleted (removed by Ubuntu Archive Robot: moved to release)
 4:15.08.1-0ubuntu1 (wily): Superseded
 4:15.08.1-0ubuntu1 (wily): Deleted (removed by Ubuntu Archive Robot: moved to release)
 ... plus 30 more; see https://launchpad.net/ubuntu/+source/baloo-widgets/+publishinghistory
OK (Y/n)? y
 * Trying to add baloo-widgets ...
baloo-widgets_4:18.08.1-1 is trying to override modified binary libkf5baloowidgets-bin_4:19.04.2-0ubuntu1. OK (y/N)? n
[New] compiz-plugins-extra_2:0.8.16-2
Previous publications in Ubuntu:
 0.9.7.0~bzr9-0ubuntu7 (quantal): Deleted (removed by Colin Watson: requested by mitya57; obsolete source package; LP: #1028363)
 0.9.7.0~bzr9-0ubuntu6 (quantal): Superseded
 0.9.7.0~bzr9-0ubuntu6 (precise): Published
 0.9.7.0~bzr9-0ubuntu5 (precise): Superseded
 0.9.7.0~bzr9-0ubuntu4 (precise): Superseded
 0.9.7.0~bzr9-0ubuntu4 (precise): Deleted (removed by Colin Watson: moved to release)
 0.9.7.0~bzr9-0ubuntu3 (precise): Superseded
 0.9.7.0~bzr9-0ubuntu2 (precise): Superseded
 0.9.7.0~bzr9-0ubuntu1 (precise): Superseded
 0.9.5.94-0ubuntu1 (precise): Superseded
 ... plus 7 more; see https://launchpad.net/ubuntu/+source/compiz-plugins-extra/+publishinghistory
OK (Y/n)? y
 * Trying to add compiz-plugins-extra ...
compiz-plugins-extra_2:0.8.16-2 is trying to override modified binary compiz-plugins-extra_1:0.9.14.0+19.04.20190223.1-0ubuntu1. OK (y/N)? n
[New] compiz-plugins-main_2:0.8.16-4
Previous publications in Ubuntu:
 1:0.9.7.0~bzr19-0ubuntu11 (quantal): Deleted (removed by Adam Conrad: Superseded by compiz)
 1:0.9.7.0~bzr19-0ubuntu10.1 (precise): Deleted (removed by Colin Watson: moved to -updates)
 1:0.9.7.0~bzr19-0ubuntu10 (quantal): Superseded
 1:0.9.7.0~bzr19-0ubuntu10 (precise): Published
 1:0.9.7.0~bzr19-0ubuntu9 (precise): Superseded
 1:0.9.7.0~bzr19-0ubuntu8 (precise): Superseded
 1:0.9.7.0~bzr19-0ubuntu7 (precise): Superseded
 1:0.9.7.0~bzr19-0ubuntu7 (precise): Deleted (removed by Colin Watson: moved to release)
 1:0.9.7.0~bzr19-0ubuntu6 (precise): Superseded
 1:0.9.7.0~bzr19-0ubuntu5 (precise): Superseded
 ... plus 31 more; see https://launchpad.net/ubuntu/+source/compiz-plugins-main/+publishinghistory
OK (Y/n)? y
 * Trying to add compiz-plugins-main ...
compiz-plugins-main_2:0.8.16-4 is trying to override modified binary compiz-plugins-main_1:0.9.14.0+19.04.20190223.1-0ubuntu1. OK (y/N)? n
[New] gcc-8-doc_8.3.0-1
No previous publications in Ubuntu
OK (Y/n)? y
 * Trying to add gcc-8-doc ...
gcc-8-doc_8.3.0-1 is trying to override modified binary gcc-8-doc_8.3.0-19ubuntu1. OK (y/N)? n
[New] google-compute-image-packages_20190124-3
No previous publications in Ubuntu
OK (Y/n)? y
 * Trying to add google-compute-image-packages ...
google-compute-image-packages_20190124-3 is trying to override modified binary python3-google-compute-engine_20190522-0ubuntu2. OK (y/N)? n
[New] libcompizconfig_2:0.8.16-2
Previous publications in Ubuntu:
 0.9.7.0~bzr428-0ubuntu7 (quantal): Deleted (removed by Adam Conrad: Superseded by compiz)
 0.9.7.0~bzr428-0ubuntu6 (quantal): Superseded
 0.9.7.0~bzr428-0ubuntu6 (precise): Published
 0.9.7.0~bzr428-0ubuntu6 (precise): Deleted (removed by Colin Watson: moved to release)
 0.9.7.0~bzr428-0ubuntu4 (precise): Superseded
 0.9.7.0~bzr428-0ubuntu3 (precise): Superseded
 0.9.7.0~bzr428-0ubuntu2 (precise): Superseded
 0.9.7.0~bzr428-0ubuntu1 (precise): Superseded
 0.9.5.94-0ubuntu2 (precise): Superseded
 0.9.5.94-0ubuntu2 (oneiric): Obsolete
 ... plus 72 more; see https://launchpad.net/ubuntu/+source/libcompizconfig/+publishinghistory
OK (Y/n)? y
 * Trying to add libcompizconfig ...
libcompizconfig_2:0.8.16-2 is trying to override modified binary libcompizconfig0-dev_1:0.9.14.0+19.04.20190223.1-0ubuntu1. OK (y/N)? n

Out-of-date BUT modified: 379 (52.28%)
Updated: 3 (0.41%)
Ubuntu Specific: 0 (0.00%)
Up-to-date [Modified]: 0 (0.00%)
Up-to-date: 0 (0.00%)
Blacklisted: 111 (15.31%)
                          -----------
Total: 725

Updating: dictem quotatool sysvbanner
Not copying packages in dry-run mode.

To post a comment you must log in.
1247. By Colin Watson

component-mismatches: fix sorting of publications to not assume comparability of Entry instances in py3

1248. By Steve Langasek

call reverse-depends with -a source instead of with -b, in order to show both reverse-build-depends and reverse-test-depends (ubuntu-dev-tools 0.174 and later)

1249. By Adam Conrad

sru-report: Ignore more linux- packages.

1250. By Łukasz Zemczak

Initial support for kernel routing lookups via KernelSeries data from the kernel team.

1251. By Andy Whitcroft

kernel_series: resync with upstream

Resync with lp:~canonical-kernel-team/+git/kteam-tools ktl/kernel_series.py.

Signed-off-by: Andy Whitcroft <email address hidden>

1252. By Steve Langasek

Merge lp:~vorlon/ubuntu-archive-tools/update-i386-whitelist

1253. By Steve Langasek

Enforce interactive confirmation when changing the i386 whitelist

1254. By Steve Langasek

Correct syntax for adding to a set

1255. By Steve Langasek

hard-code some new dependencies of lintian

1256. By Łukasz Zemczak

Add the new kernels to the exclusion list for sru-report.

1257. By Matthias Klose

 update-i386-whitelist: add gcc-10-cross and gcc-10-cross-ports

1258. By Steve Langasek

Merge lp:~xnox/ubuntu-archive-tools/unbreak-nusakan

1259. By Steve Langasek

manually exclude frei0r for now

1260. By Steve Langasek

handle the case that frei0r is already absent from the packageset (because binaries are gone from the archive, as they are now); and add spirv-tools to the bootstrap list for new glslang in -proposed.

1261. By Steve Langasek

don't include xorg in the whitelist, only the arch: all packages are used

1262. By Steve Langasek

Add ubuntu-drivers-common to the packages manually excluded from the seed

1263. By Steve Langasek

add linux-5.4 to the set, as the new source for linux-libc-dev :/

1264. By Steve Langasek

bootstrap ruby transition

1265. By Matthias Klose

change shebangs from python to python2.7

1266. By Steve Langasek

manual overrides for new -proposed build-dependencies

1267. By Steve Langasek

bootstrap boost1.71

1268. By Steve Langasek

bootstrap of new hwloc finished

1269. By Steve Langasek

linux-5.4 no longer needs special-cased

1270. By Steve Langasek

gspell bootstrap done

1271. By Steve Langasek

Merge lp:~raof/ubuntu-archive-tools/python3

1272. By Steve Langasek

gcc-10 is in-archive

1273. By Steve Langasek

bootstrap of new lintian done.

1274. By Steve Langasek

erge lp:~dannf/ubuntu-archive-tools/py3-copy-package

1275. By Colin Watson

manage-builders: add --idle filter option

1276. By Colin Watson

manage-builders: run with python3

1277. By Iain Lane

oem-metapackage-mir-check: Add script to check oem-*-meta MIR exceptions

This was requested by the MIR team. Checks the given source package against the
reference package in the Ubuntu archive (oem-qemu-meta) so that archive admins
can check for compliance with the MIR exception
(https://wiki.ubuntu.com/MIRTeam/Exceptions/OEM) and promote or NEW directly
into main.

1278. By Steve Langasek

boost1.71 bootstrap done; php7.4 needs bootstrapped now

1279. By Łukasz Zemczak

Print a warning about whenever the block proposed tag is present on any of the bugs via sru-review.

1280. By Steve Langasek

bootstrap libfido2/openssh

1281. By Steve Langasek

bootstrap tepl/gedit

1282. By Steve Langasek

bootstrap fmtlib/spdlog

1283. By Steve Langasek

include libcbor for openssh

1284. By William Grant

Port queue to Python 3.

1285. By Sebastien Bacher

Switch some of the script to use python3 so they work on focal

1286. By Steve Langasek

drop finished bootstraps

1287. By Steve Langasek

bootstrap libxcrypt

1288. By Łukasz Zemczak

Try different wording for the SRU acceptance message, adding words for including verification steps.

1289. By Łukasz Zemczak

Add new kernels to the SRU report exclusion list.

1290. By Steve Langasek

bootstrap llvm 10

1291. By Steve Langasek

bootstrap new webkit2gtk

1292. By Steve Langasek

Merge lp:~laney/ubuntu-archive-tools/mozjs68

1293. By Steve Langasek

libwpe also needed for webkit2gtk

1294. By Steve Langasek

bootstrap new bind9, lintian deps

1295. By Steve Langasek

prune whitelist for finished transitions

1296. By Colin Watson

manage-chroot: fix rollback output to include "set"

1297. By Colin Watson

manage-chroot: fix parsing of LP URLs (no mangling for those already on service host)

1298. By Steve Langasek

drop special-casing of finished bootstraps

1299. By Steve Langasek

drop special-casing of finished bootstraps

1300. By Steve Langasek

drop special-casing of finished bootstraps

1301. By Łukasz Zemczak

Add armhf+raspi and arm64+raspi to publish-image-set

1302. By Łukasz Zemczak

Ok, raspi is preinstalled as well.

1303. By Colin Watson

new-binary-debian-universe: simplify urllib imports using six

1304. By Colin Watson

new-binary-debian-universe: port to Python 3

1305. By Colin Watson

manage-chroot: port to Python 3

1306. By Steve Langasek

drop override for finished glslang bootstrap

1307. By Matthias Klose

update-i386-whitelist: add openjdk-12, required to bootstrap openjdk-13

1308. By Matthias Klose

update-i386-whitelist: add openjdk-N

1309. By Łukasz Zemczak

Add legacy-server

1310. By Steve Langasek

include riscv64 in c-m archive list

1311. By Matthias Klose

Correctly add riscv64 to components-mismatches (xnox)

1312. By Matthias Klose

focal -> groovy for devel; eoan -> focal for SRU

1313. By Colin Watson

mark-suite-dirty: port to Python 3

1314. By Łukasz Zemczak

Make branch-seeds wait a bit for the new git refs to appear before attempting to set the default_branch

1315. By Steve Langasek

branch-livefses: move to python3

1316. By Steve Langasek

branch-livefses: if the livefs already exists, just use it

1317. By Steve Langasek

branch-livefses: use specific lazr exceptions instead of 'Exception'; and be more useful when we don't have permissions to devirt a ppa.

1318. By Colin Watson

branch-livefses: convert to argparse

1319. By Colin Watson

branch-livefses: invert new vs. existing exception handling

This way round works better for the case where the user has permissions to
set require_virtualized and relative_build_score but not to actually create
the livefs (assuming somebody else has already done that), and should still
be fine, if slower, for the converse set of permissions.

1320. By Colin Watson

edit-acl: convert to argparse

1321. By Colin Watson

edit-acl: port to Python 3

1322. By Łukasz Zemczak

Switch copy-proposed-kernel to python3.

1323. By Steve Langasek

move sru-report to python3

1324. By Steve Langasek

sru-report: handle utf-8 output on stdout

1325. By Steve Langasek

Merge lp:~brian-murray/ubuntu-archive-tools/py3-sru-accept

1326. By Łukasz Zemczak

sru-remove can be switched to python 3 as well.

1327. By Steve Langasek

Merge lp:~xnox/ubuntu-archive-tools/built-using-finally

1328. By Steve Langasek

don't pass architecture=arch to apt_pkg.parse_depends, not supported on xenial.

1329. By Steve Langasek

Merge lp:~brian-murray/ubuntu-archive-tools/pu-stopped-email

1330. By Steve Langasek

use https not http for yui

1331. By Łukasz Zemczak

kernel-sru-review: look at kernel-stable-master-bug for the master bug in the description first, and only then fallback to the tags. The tags can sometimes be out-of-date.

1332. By Iain Lane

Resurrect libemail-address-xs-perl, new lintian dep

1333. By Sebastien Bacher

revert temporary change from the previous commit

1334. By Steve Langasek

Merge lp:~vorlon/ubuntu-archive-tools/clean-NBS-for-EOL

1335. By Steve Langasek

Merge lp:~brian-murray/ubuntu-archive-tools/identify-security

1336. By Steve Langasek

s/focal/groovy

1337. By Steve Langasek

Drop override for finished spdlog boostrap

1338. By Steve Langasek

Allow canonical-support as an owning team for packages in main

1339. By Steve Langasek

Fix retry-autopkgtest-regressions for compatibility with the new britney yaml schema

1340. By Steve Langasek

Fix sru-report for compatibility wit the new britney yaml schema

1341. By Steve Langasek

Merge lp:~laney/ubuntu-archive-tools/r-a-r-excuses-xz

1342. By Steve Langasek

Merge lp:~vorlon/ubuntu-archive-tools/close-EOL-bugs

1343. By Gianfranco Costamagna

modify auto-sync script to not sync if the same version has been published also in a previous Ubuntu version, not only the current devel one

Revision history for this message
Steve Langasek (vorlon) wrote :

Sorry for taking so long to review this; I looked at it briefly when it was first raised but it took some time to think through the subtleties.

I think we should NOT merge this, for a subtle reason. We currently use the extra-removals.txt in the sync-blacklist repository to document packages whose sources have been removed because they *depend on* packages that have been removed for bugginess, rather than because they are buggy themselves.

Without this change, those packages will be re-synced automatically during the next release. With this change, they will not.

Given the tradeoff between missing some of these packages which have become un-buggy and failing to resync them in the next series, and archive admins having to re-remove them in the next series if they are still broken, I think it is better to do the extra work to re-remove them, because it is worse to miss a non-buggy package in a release (or to have its re-syncing delayed, possibly causing knock-on effects) than it is to miss removing a still-unreleasable package from -proposed.

I looked at this MP briefly when it was first raised. I'm generally ok with it, but the rationale is very subtle and I think it needs to be called out here, either in a code comment or at minimum in the commit message.

Specifically, the reason it is correct to not re-sync a source package version that was previously synced and then removed is that we have an unwritten policy of only removing source packages from -proposed if that version of that package is buggy in both Debian and Ubuntu and requires sourceful changes to that package in order to be

review: Disapprove
Revision history for this message
Gianfranco Costamagna (costamagnagianfranco) wrote :

thanks Steve, I get the point, thanks for explaining, makes sense to have some extra noise but sync packages that might have been deleted in the meanwhile.

Unmerged revisions

1343. By Gianfranco Costamagna

modify auto-sync script to not sync if the same version has been published also in a previous Ubuntu version, not only the current devel one

Preview Diff

[H/L] Next/Prev Comment, [J/K] Next/Prev File, [N/P] Next/Prev Hunk
1=== modified file 'auto-sync'
2--- auto-sync 2017-12-13 19:26:54 +0000
3+++ auto-sync 2020-07-03 12:25:25 +0000
4@@ -337,21 +337,24 @@
5 return True
6
7
8-def already_in_target_series(options, difference):
9+def already_in_all_target_archive(options, difference):
10 # The published Sources files may be out of date, and if we're
11 # particularly unlucky with timing relative to a proposed-migration run
12 # it's possible for them to miss something that's in the process of
13 # being moved between pockets. To make sure, check whether an equal or
14- # higher version has already been removed from the destination archive.
15+ # higher version has already been removed from the destination archive
16+ # in all possible target series.
17 src = difference.sourcepackagename
18 from_version = difference.parent_source_version
19 to_src = version_sort_spphs(filter_pockets(
20 options.target.archive.getPublishedSources(
21- distro_series=options.target.series, source_name=src,
22+ source_name=src,
23 exact_match=True)))
24 if (to_src and
25 apt_pkg.version_compare(
26 from_version, to_src[0].source_package_version) <= 0):
27+ print("Previous publication for " + src + " with version: " + to_src[0].source_package_version
28+ + " found in Ubuntu. Not syncing version: " + from_version + " from Debian")
29 return True
30 else:
31 return False
32@@ -571,7 +574,7 @@
33 else:
34 if not published_in_source_series(options, difference):
35 continue
36- if already_in_target_series(options, difference):
37+ if already_in_all_target_archive(options, difference):
38 continue
39 print("[Updating] %s (%s [%s] < %s [%s])" % (
40 src, to_version,

Subscribers

People subscribed via source and target branches