mirrors only added to mirrors.cfg, never removed

Oh right, this is because update_mirrors.py only adds mirrors and doesn't remove them.

 13 d = feedparser.parse("https://launchpad.net/ubuntu/+archivemirrors-rss")
 14
 15 #import pprint
 16 #pp = pprint.PrettyPrinter(indent=4)
 17 #pp.pprint(d)
 18
 19 with open(sys.argv[1], "a") as outfile:
 20 for entry in d.entries:
 21 for link in entry.links:
 22 if link.href not in current_mirrors:
 23 outfile.write(link.href + "\n")

Having said that I'm not certain what ubuntu-release-upgrader would do if your mirror wasn't in that list.

I've marked an older bug as a duplicate of this one, just because.

Testing this on a Vivid system and setting the mirror to the anl.gov server caused the upgrade process to just quit. The release upgrader rewrote the entries from vivid to wily and then received 403s trying to contact the mirror, which then caused the failure.

, W:Failed to fetch http://mirror.mcs.anl.gov/pub/ubuntu/dists/wily-security/universe/binary-i386/Packages 403 Forbidden
, W:Failed to fetch http://mirror.mcs.anl.gov/pub/ubuntu/dists/wily-security/multiverse/binary-i386/Packages 403 Forbidden
, E:Some index files failed to download. They have been ignored, or old ones used instead.'. Retrying (currentRetry: 2)
2015-10-14 08:58:17,234 ERROR doUpdate() failed completely
2015-10-14 08:58:17,237 DEBUG abort called

There was no prompt to report a bug, so its hard to say how many people end up in this situation.

And then when I removed the mirror from mirrors.cfg my sources.list file was modified by the release upgrader to the following.

root@clean-vivid-amd64:/tmp/ubuntu-release-upgrader-7k_gvob6# cat /etc/apt/sources.list
# deb http://mirror.mcs.anl.gov/pub/ubuntu wily main restricted universe multiverse # disabled on upgrade to wily
# deb http://mirror.mcs.anl.gov/pub/ubuntu vivid-updates main restricted universe multiverse # disabled on upgrade to wily
# deb http://mirror.mcs.anl.gov/pub/ubuntu vivid-proposed main restricted universe multiverse
# deb http://mirror.mcs.anl.gov/pub/ubuntu vivid-security main restricted universe multiverse # disabled on upgrade to wily
# deb-src http://mirror.mcs.anl.gov/pub/ubuntu wily main restricted universe multiverse # disabled on upgrade to wily
# deb-src http://mirror.mcs.anl.gov/pub/ubuntu vivid-updates main restricted universe multiverse # disabled on upgrade to wily
# deb-src http://mirror.mcs.anl.gov/pub/ubuntu vivid-proposed main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu wily main
# deb-src http://mirror.mcs.anl.gov/pub/ubuntu vivid-security main restricted universe multiverse # disabled on upgrade to wily

I've added a branch that will also remove mirrors when updating the mirrors.cfg file, but that'll just put people in the situation where their sources.list files is modified to only only contain main which isn't good either.

I've reported the minimal sources.list issue in bug 1511783.

This bug was fixed in the package ubuntu-release-upgrader - 1:16.04.3

---------------
ubuntu-release-upgrader (1:16.04.3) xenial; urgency=medium

  * DistUpgrade/DistUpgradeController.py: if a BaseMetaPkg has no origins,
    because an Ubuntu mirror has been deactivated, then create a useful
    sources.list file that parallels the one for the deactivated mirror.
    (LP: #1511783)
  * utils/update_mirrors.py: Instead of just appending new mirrors to the
    mirrors.cfg file also remove mirrors that are no longer valid.
    (LP: #1463680)

 -- Brian Murray <email address hidden> Wed, 18 Nov 2015 11:05:19 -0800

I've tested this fix by upgrading from wily to xenial with my mirror set to mirror.mcs.anl.gov. During the upgrade process my sources.list file was changed to the following:

# deb http://mirror.mcs.anl.gov/pub/ubuntu xenial main restricted universe multiverse # disabled on upgrade to xenial
# deb http://mirror.mcs.anl.gov/pub/ubuntu xenial-updates main restricted universe multiverse # disabled on upgrade to xenial
# deb http://mirror.mcs.anl.gov/pub/ubuntu wily-proposed main restricted universe multiverse
# deb http://mirror.mcs.anl.gov/pub/ubuntu xenial-security main restricted universe multiverse # disabled on upgrade to xenial
# deb-src http://mirror.mcs.anl.gov/pub/ubuntu xenial main restricted universe multiverse # disabled on upgrade to xenial
# deb-src http://mirror.mcs.anl.gov/pub/ubuntu xenial-updates main restricted universe multiverse # disabled on upgrade to xenial
# deb-src http://mirror.mcs.anl.gov/pub/ubuntu wily-proposed main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu xenial main
deb http://archive.ubuntu.com/ubuntu xenial main restricted universe multiverse # auto generated by ubuntu-release-upgrader
deb http://archive.ubuntu.com/ubuntu xenial-updates main restricted universe multiverse # auto generated by ubuntu-release-upgrader
deb http://archive.ubuntu.com/ubuntu xenial-security main restricted universe multiverse # auto generated by ubuntu-release-upgrader
deb-src http://archive.ubuntu.com/ubuntu xenial main restricted universe multiverse # auto generated by ubuntu-release-upgrader
deb-src http://archive.ubuntu.com/ubuntu xenial-updates main restricted universe multiverse # auto generated by ubuntu-release-upgrader
deb-src http://archive.ubuntu.com/ubuntu xenial-security main restricted universe multiverse # auto generated by ubuntu-release-upgrader
# deb-src http://mirror.mcs.anl.gov/pub/ubuntu xenial-security main restricted universe multiverse # disabled on upgrade to xenial

So this, and the fix for bug 1511783, looks good to me.