UIFE - remove ubiquity upgrade option

Mar 19 11:59:14 ubuntu ubiquity: Traceback (most recent call last):
Mar 19 11:59:14 ubuntu ubiquity: File "/usr/bin/apt-clone", line 148, in <module>
Mar 19 11:59:14 ubuntu ubiquity: args.source, args.destination, new_distro=args.new_distro_codename, protect_installed=protect_installed)
Mar 19 11:59:14 ubuntu ubiquity: File "/usr/lib/python3/dist-packages/apt_clone.py", line 501, in restore_state
Mar 19 11:59:14 ubuntu ubiquity: self._restore_package_selection(statefile, targetdir, protect_installed, exclude_pkgs)
Mar 19 11:59:14 ubuntu ubiquity: File "/usr/lib/python3/dist-packages/apt_clone.py", line 658, in _restore_package_selection
Mar 19 11:59:14 ubuntu ubiquity: cache.commit(self.fetch_progress, self.install_progress)
Mar 19 11:59:14 ubuntu ubiquity: File "/usr/lib/python3/dist-packages/apt/cache.py", line 529, in commit
Mar 19 11:59:14 ubuntu ubiquity: raise SystemError("installArchives() failed")
Mar 19 11:59:14 ubuntu ubiquity: SystemError: installArchives() failed
Mar 19 11:59:14 ubuntu /plugininstall.py: log-output -t ubiquity chroot /target umount /sys
Mar 19 11:59:14 ubuntu /plugininstall.py: log-output -t ubiquity chroot /target umount /proc
Mar 19 11:59:14 ubuntu /plugininstall.py: log-output -t ubiquity umount /target/run
Mar 19 11:59:14 ubuntu /plugininstall.py: log-output -t ubiquity umount /target/dev
Mar 19 11:59:14 ubuntu /plugininstall.py: log-output -t ubiquity umount -f /target/proc
Mar 19 11:59:14 ubuntu /plugininstall.py: log-output -t ubiquity umount -f /target/sys
Mar 19 11:59:14 ubuntu /plugininstall.py: log-output -t ubiquity umount -f /target/dev
Mar 19 11:59:14 ubuntu /plugininstall.py: log-output -t ubiquity umount -f /target/run
Mar 19 11:59:14 ubuntu /plugininstall.py: Could not restore packages from the previous install:
Mar 19 11:59:14 ubuntu /plugininstall.py: Traceback (most recent call last):
Mar 19 11:59:14 ubuntu /plugininstall.py: File "/usr/share/ubiquity/plugininstall.py", line 246, in run
Mar 19 11:59:14 ubuntu /plugininstall.py: self.apt_clone_restore()
Mar 19 11:59:14 ubuntu /plugininstall.py: File "/usr/share/ubiquity/plugininstall.py", line 1486, in apt_clone_restore
Mar 19 11:59:14 ubuntu /plugininstall.py: restore_cmd, preexec_fn=install_misc.debconf_disconnect)
Mar 19 11:59:14 ubuntu /plugininstall.py: File "/usr/lib/python3.6/subprocess.py", line 291, in check_call
Mar 19 11:59:14 ubuntu /plugininstall.py: raise CalledProcessError(retcode, cmd)
Mar 19 11:59:14 ubuntu /plugininstall.py: subprocess.CalledProcessError: Command '['apt-clone', 'restore-new-distro', '/target/ubiquity-apt-clone/apt-clone-state-ubuntu.tar.gz', 'bionic', '--destination', '/target']' returned non-zero exit status 1.

and earlier in syslog

Mar 19 11:59:06 ubuntu ubiquity: Chrooting into
Mar 19 11:59:06 ubuntu ubiquity: /target/
Mar 19 11:59:06 ubuntu ubiquity:
Mar 19 11:59:06 ubuntu ubiquity: dpkg: error: cannot stat pathname '/tmp/apt-dpkg-install-Kthb4J': No such file or directory

Status changed to 'Confirmed' because the bug affects multiple users.

From the duplicate report:

It can also be reproduce with the following command
$ sudo apt-clone restore-new-distro \
    /target/ubiquity-apt-clone/apt-clone-state-ubuntu.tar.gz \
    bionic --destination /target

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/1756862

The traceback from apt-clone can be seen here:

Traceback (most recent call last):
  File "/usr/bin/apt-clone", line 148, in <module>
    args.source, args.destination, new_distro=args.new_distro_codename, protect_installed=protect_installed)
  File "/usr/lib/python3/dist-packages/apt_clone.py", line 501, in restore_state
    self._restore_package_selection(statefile, targetdir, protect_installed, exclude_pkgs)
  File "/usr/lib/python3/dist-packages/apt_clone.py", line 656, in _restore_package_selection
    self._restore_package_selection_in_cache(statefile, cache, protect_installed, exclude_pkgs)
  File "/usr/lib/python3/dist-packages/apt_clone.py", line 633, in _restore_package_selection_in_cache
    resolver.resolve()
apt_pkg.Error: E:Unable to correct problems, you have held broken packages.

Running it with --simulate we can see something weird is going on:

sudo APT_CLONE_DEBUG_RESOLVER=1 apt-clone restore-new-distro /tmp/apt-clone-state-ubuntu.tar.gz bionic --destination / --simulate
missing: adwaita-icon-theme-full,fonts-cantarell,fonts-nanum,fonts-takao-pgothic,gir1.2-totem-plparser-1.0,gnome-session,gnome-themes-standard,gnome-themes-standard-data,gnome-user-share,gpart,gtk2-engines-pixbuf,ipython3,kerneloops-applet,libboost-date-time1.62.0,libboost-filesystem1.62.0,libboost-iostreams1.62.0,libboost-system1.62.0,libboost-thread1.62.0,libcanberra-gtk-module,libcanberra-gtk0,libcapnp-0.5.3,libcdio-cdda1,libcdio-paranoia1,libcdio13,libdfu1,libfreerdp-cache1.1,libfreerdp-client1.1,libfreerdp-codec1.1,libfreerdp-common1.1.0,libfreerdp-core1.1,libfreerdp-crypto1.1,libfreerdp-gdi1.1,libfreerdp-locale1.1,libfreerdp-plugins-standard,libfreerdp-primitives1.1,libfreerdp-utils1.1,libfwupd1,libgom-1.0-common,libicu57,liblouis12,libntfs-3g872,libpython-stdlib,libwinpr-crt0.1,libwinpr-dsparse0.1,libwinpr-environment0.1,libwinpr-file0.1,libwinpr-handle0.1,libwinpr-heap0.1,libwinpr-input0.1,libwinpr-interlocked0.1,libwinpr-library0.1,libwinpr-path0.1,libwinpr-
pool0.1,libwinpr-registry0.1,libwinpr-rpc0.1,libwinpr-sspi0.1,libwinpr-synch0.1,libwinpr-sysinfo0.1,libwinpr-thread0.1,libwinpr-utils0.1,linux-headers-4.13.0-16,linux-headers-4.13.0-16-generic,linux-headers-4.13.0-21,linux-headers-4.13.0-21-generic,linux-headers-4.13.0-38,linux-headers-4.13.0-38-generic,linux-image-4.13.0-16-generic,linux-image-4.13.0-21-generic,linux-image-4.13.0-38-generic,linux-image-extra-4.13.0-16-generic,linux-image-extra-4.13.0-21-generic,linux-image-extra-4.13.0-38-generic,python,python-minimal,python-pexpect,python-ptyprocess,python2.7,python2.7-minimal,python3-decorator,python3-ipdb,python3-ipython,python3-ipython-genutils,python3-pickleshare,python3-prompt-toolkit,python3-pygments,python3-simplegeneric,python3-traitlets,python3-wcwidth,tcpd,ubuntu-gnome-desktop

Some of those packages are in main. I was able to workaround this by setting "allow-insecure=true" in the sources.list file in the apt-clone file, but that is not a solution.

I think I was encountering a different problem yesterday as I was trying to ubiquity upgrade an old (not up to date) version of artful. After updating my artful system and then trying the upgrade I'm encountering the error message:

"dpkg: error: cannot stat pathname '/tmp/apt-dpkg-install-7IdNNM': No such file or directory"

This is the one the original reporter experienced.

After modifying apt_clone.py to set "Debug::pkgDPkgPM" I saw the error messages in the attached screenshot.

Another screenshot with different debugging set.

Looking around at other ubiquity bugs we can see that apt-clone with the restore-new-distro option has not worked for lots of people for quite some time. As an example bug see bug 1152399 which is a collection of duplicate bugs where the "install" of Ubuntu failed when calculating the upgrade from a previous release to a later release of Ubuntu. This leaves people in a hard to resolve situation and subsequently due to that issue and this crash with apt-clone we have decided to remove the option to upgrade a release of Ubuntu from ubiquity.

This bug was fixed in the package ubiquity - 18.04.6

---------------
ubiquity (18.04.6) bionic; urgency=medium

  [ Steve Langasek ]
  * If we are running ubiquity-dm on VT1, we must conflict with
    <email address hidden> and not rely on some other display manager to do
    this for us.

  [ Brian Murray ]
  * Drop the option to upgrade from an old release using ubiquity as it is
    crashing and leaves users in a broken state. (LP: #1756862)

  [ litianzhi ]
  * Fix untranslated "This software is subject to license" (LP: #1762097)

  [ Mathieu Trudel-Lapierre ]
  * debian/real-po/, po/: Translations import from Launchpad.

 -- Mathieu Trudel-Lapierre <email address hidden> Fri, 13 Apr 2018 15:54:03 -0400

I've emailed the ubuntu-doc team about this change, https://lists.ubuntu.com/archives/ubuntu-doc/2018-April/020601.html.

Post-hoc approval of the UI Freeze exception, per the understanding that this does not impact the existing docs.