Merge ~xnox/britney/+git/britney2-ubuntu:ignore-di into ~ubuntu-release/britney/+git/britney2-ubuntu:master

[Dimitri John Ledkov]

suiteloader: stop loading debian-installer directories

udebs do not have symbol versioned shlibdeps, thus it means that when
there is glibc in proposed, any packages built in proposed get a udeb
depends on the strictly higher version of glibc in proposed. This
leads to entaglement of the $world with glibc transition. Similar
story with glib, openssl, etc.

However, we do not care about installability of debian-installer udebs
as they are obsolete in $devel series. We do care about them in the
supported stable lts series, however we currently do not use britney
for SRU releases there and usually we do migrate all the things,
before building point releases.

If there is need for britney checks on udebs in supported stable lts
series, loading udebs must become a conditional .conf option. But I
will need help with that, as I can't seem to find the production
configs used for the supported lts series and how to treak them
differently from $devel.

[Steve Langasek (vorlon)]

In terms of being able to merge this back to Debian, it would be preferable to make this a configurable option rather than deleting the code.

[Iain Lane (laney)]

Right, this should be configurable please, so that (1) it can be done differently for stable releases, (2) it can be upstreamed - which should ideally be done ASAP if & when we merge something here.

See britney1 make_b2_config for the config file generation. However, for testing, you can ignore b1 and use a static configuration.

Question: would it be better to stop building udebs at all (noudeb profile, I'm thinking like we do for nocheck on riscv64)?

review: Needs Fixing

[Dimitri John Ledkov (xnox)]

@laney

noudeb is very hard, and yes i want to build with noudeb by default, but it is pain to get there. making britney ignore udebs is a very helpful tool to get there.

1) lp doesn't set build-profiles. also launchpad doesn't like when it dispatches a build, and it produces no debs (builds appear as failed, as nothing to build for us, because profiles result in nothing to build)

2) in dpkg, one can make noudeb implicitely declared one, but if one does that nochange rebuilds of all the udeb carrying packages must be uploaded and migrated simultaneously as otherwise britney will not let them migrate

3) there are a lot of dependencies between all the udebs. For example, i'm trying to unwind all the dependencies on *-modules, to be able to stop building kernel udebs. It will take many more package removals and delta in the packaging to eradicate all other udebs.

Thus yes, I hope to patch all the things to support noudeb build profile, and activate it by default. But to get there I would prefer to be able to drop udebs one by one, without stalling migrations and/or making it a mega-transition.

d9891a7... by Dimitri John Ledkov on 2021-02-18

suiteloader: add configuration option whether to load udebs.

udebs do not have symbol versioned shlibdeps, thus it means that when
there is glibc in proposed, any packages built in proposed get a udeb
depends on the strictly higher version of glibc in proposed. This
leads to entaglement of the $world with glibc transition. Similar
story with glib, openssl, etc.

By setting the new configuration option to HAS_UDEBS = no, one can
make suiteloader not look at udebs, and thus use britney even when
installability of udebs is not required.

Unmerged commits

d9891a7... by Dimitri John Ledkov on 2021-02-18

suiteloader: add configuration option whether to load udebs.

udebs do not have symbol versioned shlibdeps, thus it means that when
there is glibc in proposed, any packages built in proposed get a udeb
depends on the strictly higher version of glibc in proposed. This
leads to entaglement of the $world with glibc transition. Similar
story with glib, openssl, etc.

By setting the new configuration option to HAS_UDEBS = no, one can
make suiteloader not look at udebs, and thus use britney even when
installability of udebs is not required.

diff --git a/britney.conf b/britney.conf
index 74ecd4b..62403bd 100644
--- a/britney.conf
+++ b/britney.conf
@@ -44,6 +44,9 @@ BREAK_ARCHES      =
 # if you're in this list, you are a new architecture
 NEW_ARCHES        =
 
+# should udebs be looked at?
+HAS_UDEBS         = yes
+
 # priorities and delays
 MINDAYS_LOW       = 0
 MINDAYS_MEDIUM    = 0
diff --git a/britney2/inputs/suiteloader.py b/britney2/inputs/suiteloader.py
index 964f239..0208ecc 100644
--- a/britney2/inputs/suiteloader.py
+++ b/britney2/inputs/suiteloader.py
@@ -489,22 +489,23 @@ class DebMirrorLikeSuiteContentLoader(SuiteContentLoader):
                                             binary_dir,
                                             'Packages')
                     filename = possibly_compressed(filename)
-                    udeb_filename = os.path.join(basedir,
-                                                 component,
-                                                 "debian-installer",
-                                                 binary_dir,
-                                                 "Packages")
-                    # We assume the udeb Packages file is present if the
-                    # regular one is present
-                    udeb_filename = possibly_compressed(udeb_filename)
                     self._read_packages_file(filename,
                                              arch,
                                              suite.sources,
                                              packages)
-                    self._read_packages_file(udeb_filename,
-                                             arch,
-                                             suite.sources,
-                                             packages)
+                    if getattr(self._base_config, 'has_udebs', 'yes') == 'yes':
+                        udeb_filename = os.path.join(basedir,
+                                                     component,
+                                                     "debian-installer",
+                                                     binary_dir,
+                                                     "Packages")
+                        # We assume the udeb Packages file is present if the
+                        # regular one is present
+                        udeb_filename = possibly_compressed(udeb_filename)
+                        self._read_packages_file(udeb_filename,
+                                                 arch,
+                                                 suite.sources,
+                                                 packages)
                 # create provides
                 provides = create_provides_map(packages)
                 binaries[arch] = packages