Fails to detect package download errors on architectures other than amd64

Bug #998492 reported by Ekkachai Leelaprasertsin
186
This bug affects 22 people
Affects Status Importance Assigned to Milestone
Baltix
New
Undecided
Unassigned
ubiquity (Ubuntu)
Fix Released
High
Colin Watson
Precise
Fix Released
High
Colin Watson

Bug Description

TEST CASE: Install an i386 system (importantly, must not be amd64) whose networking goes through a deliberately broken proxy; for example, comment #5 has an attachment that can be modified to break packages with selected names. If one of those packages is a language support package - firefox-locale-en is usually a good victim for this kind of thing - then ubiquity will crash. The fix arranges for it to skip installing language packs and carry on instead, which was how it works in 12.04 on amd64 but was broken on other architectures.

Original report follows:

Installment process, many errors occur about copy files.

ProblemType: Bug
DistroRelease: Ubuntu 12.04
Package: ubiquity 2.10.16
ProcVersionSignature: Ubuntu 3.2.0-23.36-generic-pae 3.2.14
Uname: Linux 3.2.0-23-generic-pae i686
ApportVersion: 2.0.1-0ubuntu5
Architecture: i386
CasperVersion: 1.315
Date: Sat May 12 23:53:20 2012
InstallCmdLine: file=/cdrom/preseed/ubuntu.seed boot=casper initrd=/casper/initrd.lz quiet splash -- maybe-ubiquity
LiveMediaBuild: Ubuntu 12.04 LTS "Precise Pangolin" - Release i386 (20120423)
ProcEnviron:
 LANGUAGE=en_US.UTF-8
 TERM=linux
 PATH=(custom, no user)
 LANG=en_US.UTF-8
SourcePackage: ubiquity
UpgradeStatus: No upgrade log present (probably fresh install)

Revision history for this message
Ekkachai Leelaprasertsin (berry99) wrote :
Revision history for this message
Ubuntu Foundations Team Bug Bot (crichton) wrote : Traceback

Exception during installation:
May 12 16:50:50 ubuntu plugininstall.py: SystemError: E:Sub-process /usr/bin/dpkg returned an error code (1)
May 12 16:50:50 ubuntu plugininstall.py:

tags: added: installer-crash
Revision history for this message
Brian Murray (brian-murray) wrote : Re: Error while copying OS files

A package that was being installed from the internet became corrupted on disk or in memory:

May 12 16:50:08 ubuntu ubiquity: dpkg-deb (subprocess): data: internal bzip2 read error: 'DATA_ERROR'
May 12 16:50:08 ubuntu ubiquity: dpkg-deb: error: subprocess <decompress> returned error exit status 2
May 12 16:50:08 ubuntu ubiquity: dpkg: error processing /target/var/cache/apt/archives/libreoffice-core_1%3a3.5.3-0ubuntu1_i386.deb (--unpack):
May 12 16:50:08 ubuntu ubiquity: short read on buffer copy for backend dpkg-deb during `./usr/lib/libreoffice/program/libsvxcorelo.so'

Revision history for this message
Colin Watson (cjwatson) wrote :

This should have been caught by previous work, but wasn't due to:

May 12 16:49:13 ubuntu plugininstall.py: Failed to find package object for /target/var/cache/apt/archives/libreoffice-core_1%3a3.5.3-0ubuntu1_i386.deb: "The cache has no package named 'libreoffice-core:i386'"

Changed in ubiquity (Ubuntu):
status: New → Triaged
importance: Undecided → High
assignee: nobody → Colin Watson (cjwatson)
Revision history for this message
Colin Watson (cjwatson) wrote :
Changed in ubiquity (Ubuntu Precise):
status: New → Triaged
importance: Undecided → High
assignee: nobody → Colin Watson (cjwatson)
milestone: none → ubuntu-12.04.1
Colin Watson (cjwatson)
description: updated
Colin Watson (cjwatson)
Changed in ubiquity (Ubuntu):
status: Triaged → Fix Committed
Changed in ubiquity (Ubuntu Precise):
status: Triaged → In Progress
summary: - Error while copying OS files
+ Fails to detect package download errors on architectures other than
+ amd64
Revision history for this message
Launchpad Janitor (janitor) wrote :
Download full text (6.3 KiB)

This bug was fixed in the package ubiquity - 2.11.0

---------------
ubiquity (2.11.0) quantal; urgency=low

  [ Colin Watson ]
  * Install oem-config-slideshow-ubuntu in a separate pass from
    oem-config-$frontend and ubiquity-frontend-$frontend, since it may be
    missing from images; and only do this for the GTK frontend in any case,
    since other frontends don't currently use the slideshow (LP: #987050).
  * Re-enable alpha warning for quantal.
  * Upgrade to debhelper 9 for improved handling of compiler flags. Make
    sure that /usr/lib/girepository-1.0/ stays where it's supposed to be
    despite changes in the default libdir for multiarch.
  * Port to Python 3:
    - Use Python 3-style print functions.
    - Use "except Exception as e" syntax rather than the old-style "except
      Exception, e".
    - Use reduce from functools rather than relying on the builtin.
    - Use list comprehensions rather than filter or map.
    - Use open() rather than file().
    - Import configparser rather than ConfigParser if available.
    - Use input() rather than raw_input() when running under Python 3.
    - Use set comprehensions.
    - Import quote from urllib.parse rather than urllib if available.
    - Use new-style octal literals.
    - Use test.support rather than test.test_support if available.
    - Add --python2 and --python3 options to tests/run to force running the
      tests under Python 2 or 3 respectively.
    - Handle renaming of __builtin__ to builtins in Python 3.
    - Only pass unicode=1 to gettext.install in Python 2.
    - Port ubiquity.auto_update to python-apt 0.8 progress classes.
    - Use Python 3 names for itertools.izip and itertools.izip_longest if
      available.
    - Use helpers from the six module to deal with some bytes/unicode
      differences.
    - As a general rule, open subprocesses with universal_newlines=True when
      expecting to read text from them. This has no effect on Python 2
      (aside from \r\n conversion and the like, which is mostly a no-op for
      us), but causes Python 3 to read str rather than bytes. The
      exceptions at the moment are: debconf-copydb subprocesses, which
      return mixed-encoding data that needs to be handled specially; and
      when feeding the output of a subprocess to hashlib.
    - Use six.reraise rather than the three-argument form of raise.
    - Adjust test_filteredcommand for Python 3 text handling.
    - Fix test_ubi_partman.question_has_variables to handle templates files
      as binary data, since they're mixed-encoding.
    - Cope with assertItemsEqual/assertCountEqual naming difference between
      Python 2.7 and 3.2.
    - Rearrange ubiquity.i18n.get_translations to treat debconf-copydb
      output as binary data and do field-dependent decoding.
    - Adjust test_misc and test_upower to cope with file type changes in
      Python 3.
    - Use xml.etree.cElementTree instead of libxml2; it's faster, has a
      smaller footprint by virtue of being built into the standard library,
      arguably easier to read, and works with Python 3.
    - Fix a slew of file handle leaks, including making much more liberal
      use of context managers.
   ...

Read more...

Changed in ubiquity (Ubuntu):
status: Fix Committed → Fix Released
tags: added: bugpattern-written
Revision history for this message
Clint Byrum (clint-fewbar) wrote : Please test proposed package

Hello Ekkachai, or anyone else affected,

Accepted ubiquity into precise-proposed. The package will build now and be available in a few hours. Please test and give feedback here. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Thank you in advance!

Changed in ubiquity (Ubuntu Precise):
status: In Progress → Fix Committed
tags: added: verification-needed
Changed in ubiquity (Ubuntu):
assignee: Colin Watson (cjwatson) → Sterling Butters (sterlingbutters)
Revision history for this message
Timothy Jon Terry (timbyt) wrote : Re: [Bug 998492] Re: Fails to detect package download errors on architectures other than amd64

I will send this too You, I stalled for the fourth time on a thoroughly
formatted drive...I disabled the install updates while installing the os.
Then once loaded I purposely waited a couple of days running to load the
Java programs ...seem pretty stable. I did disable Hyperthreading, the
front end audio also in the Bios and also the networking and legacy USB
load ins in the BIOS ...the system work very well so far!

On Sun, Jun 17, 2012 at 6:26 PM, Sterling Butters <<email address hidden>
> wrote:

> ** Changed in: ubiquity (Ubuntu)
> Assignee: Colin Watson (cjwatson) => Sterling Butters (sterlingbutters)
>
> --
> You received this bug notification because you are subscribed to a
> duplicate bug report (1009316).
> https://bugs.launchpad.net/bugs/998492
>
> Title:
> Fails to detect package download errors on architectures other than
> amd64
>
> Status in “ubiquity” package in Ubuntu:
> Fix Released
> Status in “ubiquity” source package in Precise:
> Fix Committed
> Status in Baltix GNU/Linux:
> New
>
> Bug description:
> TEST CASE: Install an i386 system (importantly, must not be amd64)
> whose networking goes through a deliberately broken proxy; for
> example, comment #5 has an attachment that can be modified to break
> packages with selected names. If one of those packages is a language
> support package - firefox-locale-en is usually a good victim for this
> kind of thing - then ubiquity will crash. The fix arranges for it to
> skip installing language packs and carry on instead, which was how it
> works in 12.04 on amd64 but was broken on other architectures.
>
> Original report follows:
>
> Installment process, many errors occur about copy files.
>
> ProblemType: Bug
> DistroRelease: Ubuntu 12.04
> Package: ubiquity 2.10.16
> ProcVersionSignature: Ubuntu 3.2.0-23.36-generic-pae 3.2.14
> Uname: Linux 3.2.0-23-generic-pae i686
> ApportVersion: 2.0.1-0ubuntu5
> Architecture: i386
> CasperVersion: 1.315
> Date: Sat May 12 23:53:20 2012
> InstallCmdLine: file=/cdrom/preseed/ubuntu.seed boot=casper
> initrd=/casper/initrd.lz quiet splash -- maybe-ubiquity
> LiveMediaBuild: Ubuntu 12.04 LTS "Precise Pangolin" - Release i386
> (20120423)
> ProcEnviron:
> LANGUAGE=en_US.UTF-8
> TERM=linux
> PATH=(custom, no user)
> LANG=en_US.UTF-8
> SourcePackage: ubiquity
> UpgradeStatus: No upgrade log present (probably fresh install)
>
> To manage notifications about this bug go to:
>
> https://bugs.launchpad.net/ubuntu/+source/ubiquity/+bug/998492/+subscriptions
>

Colin Watson (cjwatson)
Changed in ubiquity (Ubuntu):
assignee: Sterling Butters (sterlingbutters) → Colin Watson (cjwatson)
Revision history for this message
Stéphane Graber (stgraber) wrote :

So, I just spent an hour or so trying to test that fix, it proved to be a lot trickier than expected :)

Reproducing the issue was easy using that perl script, the problem was to test the fix without getting into other crashes...

I "think" that the fix works, in the sense that I now correctly see the packages being downloaded and no ubiquity crashes happening during that time, however, that perl script triggers the following issues:
 - Ubiquity crashes on invalid geoip information (had to disable the proxy to have it go past that point)
 - Ubiquity crashes on invalid apt list ("Encountered a section with no Package: header")

As I mentioned, I managed to workaround the first crash by simply bypassing the proxy for that step but that second crash is a bit more annoying though I "believe" is not a regression from this fix and is a corner case I hit with my testing environment.

To enforce the use of the proxy, I set it up as a transparent http proxy, thereby forcing every tcp/80 traffic through it, this very likely catches a lot more than what was used by the reporter and can explain why I'm getting more crashes.

tags: added: verification-done
removed: verification-needed
Revision history for this message
Launchpad Janitor (janitor) wrote :

This bug was fixed in the package ubiquity - 2.10.17

---------------
ubiquity (2.10.17) precise-proposed; urgency=low

  [ Colin Watson ]
  * Improve download verification to handle systems not configured for
    multiarch, i.e. anything other than amd64 by default (LP: #998492).

  [ Evan Dandrea ]
  * Remove migration-assistant following foundations-q-testing-
    migration-assistant. Thanks Dmitrijs Ledkovs!
 -- Evan Dandrea <email address hidden> Fri, 25 May 2012 16:49:43 +0100

Changed in ubiquity (Ubuntu Precise):
status: Fix Committed → Fix Released
Sam K (sam230)
Changed in ubiquity (Ubuntu):
assignee: Colin Watson (cjwatson) → nobody
Colin Watson (cjwatson)
Changed in ubiquity (Ubuntu):
assignee: nobody → Colin Watson (cjwatson)
Revision history for this message
aimwin (aim-mail) wrote :

Crash after key in user id and password.

I have full functioning and full update Ubuntu 12.04 on external usb hard disk Fujisu 400 GB.
I installed ubiguity to the installed Ubuntu 12.04.
Reboot, and boot into Ubuntu 1204 and start installation to the main notebook hard disk sda13 (seagate 300 gb)

It crashed.

Revision history for this message
aimwin (aim-mail) wrote :

Try installation again and exact crash at the end after userid and password config.

Revision history for this message
Lachlan Andrew (lachlan-andrew) wrote :

I was directed to this bug report by the installation of lubuntu 13.10 and lubuntu 14.04 on an IBM X40. The bug does not seem relevant, and so I plan to file a separate bug report. However, I'm posting this here to let you know that there is a bug in the system that detects which bug has occurred during an install...

The end of /var/log/syslog is:
Dec 29 00:56:36 lubuntu ubiquity: Unpacking upstart-bin (from .../upstart-bin_1.13.2-0ubuntu2_i386.deb) ...
Dec 29 00:56:36 lubuntu ubiquity: dpkg: error processing /cdrom//pool/main/u/upstart/upstart-bin_1.13.2-0ubuntu2_i386.deb (--unpack):
Dec 29 00:56:36 lubuntu ubiquity: trying to overwrite '/etc/bash_completion.d/upstart', which is also in package upstart 1.13.2-0ubuntu2
Dec 29 00:56:36 lubuntu ubiquity: dpkg-deb: error: subprocess paste was killed by signal (Broken pipe)
Dec 29 00:56:37 lubuntu ubiquity: Processing triggers for man-db ...
Dec 29 00:56:47 lubuntu ubiquity: Errors were encountered while processing:
Dec 29 00:56:47 lubuntu ubiquity: /cdrom//pool/main/u/upstart/upstart-bin_1.13.2-0ubuntu2_i386.deb
Dec 29 00:56:48 lubuntu ubiquity: Error in function:
Dec 29 00:56:48 lubuntu ubiquity:
Dec 29 00:56:48 lubuntu /plugininstall.py: Exception during installation:
Dec 29 00:56:48 lubuntu /plugininstall.py: SystemError: E:Sub-process /usr/bin/dpkg returned an error code (1)
Dec 29 00:56:48 lubuntu /plugininstall.py:

John Woods (jmwoods1970)
affects: ubiquity (Ubuntu Precise) → lliurex-ubiquity-casper (Ubuntu Precise)
Revision history for this message
Ubuntu QA Website (ubuntuqa) wrote :

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/998492

tags: added: iso-testing
William Grant (wgrant)
affects: lliurex-ubiquity-casper (Ubuntu) → ubuntu
affects: ubuntu → ubiquity (Ubuntu)
Revision history for this message
AllesMeins (spam-startrekarchiv) wrote :

Just had the 20.04 Beta installation crashing on me and was sent here by the crash reporter claiming that this bug had already been filed?!?

I did a "minimal install" on an amd64 system and the installer just crashed near the end (installing packages). Unfortunatly it didn't display any error besides the information that it just crashed and there is no error-report stored in /var/crash - so I can't tell what exactly the error was and why I was sended here. I'm attaching the dkpg and installer log - maybe you can find something usefull in there.

Revision history for this message
AllesMeins (spam-startrekarchiv) wrote :
Revision history for this message
AllesMeins (spam-startrekarchiv) wrote :
Revision history for this message
Fábio Lima (fhl) wrote :

The same happens to me as ins #15. Anda I this should not happen.
I'm installing ubuntu 20.04 beta amd64.

Revision history for this message
AllesMeins (spam-startrekarchiv) wrote :

I did some testing and for me the problem only appeared when selecting "install third party packages" during install. Without the install went fine.

To post a comment you must log in.
This report contains Public information  
Everyone can see this information.

Other bug subscribers

Remote bug watches

Bug watches keep track of this bug in other bug trackers.