ubuntu release upgrade failed due to too many open files

Status changed to 'Confirmed' because the bug affects multiple users.

Here is the error from main.log:

2013-08-19 20:34:44,337 DEBUG quirks: running PreCacheOpen
2013-08-19 20:34:44,337 DEBUG running Quirks.PreCacheOpen
2013-08-19 20:34:44,968 DEBUG failed to lock the cache, retrying (1)
2013-08-19 20:34:45,584 DEBUG failed to lock the cache, retrying (2)
2013-08-19 20:34:46,199 DEBUG failed to lock the cache, retrying (3)
2013-08-19 20:34:46,794 DEBUG failed to lock the cache, retrying (4)
2013-08-19 20:34:47,487 DEBUG failed to lock the cache, retrying (5)
2013-08-19 20:34:48,090 DEBUG failed to lock the cache, retrying (6)
2013-08-19 20:34:48,726 DEBUG failed to lock the cache, retrying (7)
2013-08-19 20:34:49,370 DEBUG failed to lock the cache, retrying (8)
2013-08-19 20:34:50,027 DEBUG failed to lock the cache, retrying (9)
2013-08-19 20:34:50,631 DEBUG failed to lock the cache, retrying (10)
2013-08-19 20:34:51,266 DEBUG failed to lock the cache, retrying (11)
2013-08-19 20:34:51,905 DEBUG failed to lock the cache, retrying (12)
2013-08-19 20:34:52,568 DEBUG failed to lock the cache, retrying (13)
2013-08-19 20:34:53,201 DEBUG failed to lock the cache, retrying (14)
2013-08-19 20:34:53,848 DEBUG failed to lock the cache, retrying (15)
2013-08-19 20:34:54,643 DEBUG failed to lock the cache, retrying (16)
2013-08-19 20:34:55,131 ERROR openCache() failed: 'E:Could not open file /var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_raring_multiverse_i18n_Translation-en - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_raring_main_i18n_Translation-en - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_raring_multiverse_binary-i386_Packages - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_raring_universe_binary-i386_Packages - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_raring_restricted_binary-i386_Packages - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_raring_main_binary-i386_Packages - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_raring_multiverse_binary-amd64_Packages - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_raring_universe_binary-amd64_Packages - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_raring_restricted_binary-amd64_Packages - open (24: Too many open files), E:Could not open file /var/lib/apt/lists/us.archive.ubuntu.com_ubuntu_dists_raring_main_binary-amd64_Packages - open (24: Too many open files)'
2013-08-19 20:34:55,131 ERROR self.prepared() failed

This seems to be an issue with how the apt cache is managed in DistUpgradeController.py.

A solution for a similar issue in some other packages can be found in bug 1051935.

Most of the users encountering this bug have a few PPAs enabled which will cause their own issues during the upgrade process, so I'm downgrading the importance to Medium.

Hello Kevin, or anyone else affected,

Accepted ubuntu-release-upgrader into saucy-proposed. The package will build now and be available at http://launchpad.net/ubuntu/+source/ubuntu-release-upgrader/1:0.205.2 in a few hours, and then in the -proposed repository.

Please help us by testing this new package. See https://wiki.ubuntu.com/Testing/EnableProposed for documentation how to enable and use -proposed. Your feedback will aid us getting this update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug, mentioning the version of the package you tested, and change the tag from verification-needed to verification-done. If it does not fix the bug for you, please add a comment stating that, and change the tag to verification-failed. In either case, details of your testing will help us make a better decision.

Further information regarding the verification process can be found at https://wiki.ubuntu.com/QATeam/PerformingSRUVerification . Thank you in advance!

This bug was fixed in the package ubuntu-release-upgrader - 1:0.208

---------------
ubuntu-release-upgrader (1:0.208) trusty; urgency=low

  [ Brian Murray ]
  * Synchronize the list of metapackages used in the code. LP: #1225016.
  * Properly display error messages encountered by DistUpgradeFetcherCore.
    (LP: #1241660)
  * DistUpgrade/DistUpgradeCache.py: Do not initialize the cache until we know
    it is unlocked. (LP: #1214186)
 -- Brian Murray <email address hidden> Fri, 25 Oct 2013 09:17:56 -0700

Using the ubuntu-release-upgrader from saucy-proposed I successfully completed an upgrade from raring to saucy. However, I had not set the ulimit, this was just to confirm that the upgrade still works.

I tested the release upgrader from -proposed using update-manager -p after having set a ulimit of 256 and running synaptic at the same time. I did receive a dialog regarding having another package manager running.

Sorry.. I am just a Chinese student and my English is not very well. So I do not know what you're talking about (You succeeded to update it?). And I have tried to update it again that day and then I succeeded to upgrade it.

Anyway, I am very grateful for your E-mail.

Still broken for me. I enabled saucy-proposed, 'sudo apt-get update' and then 'sudo do-release-upgrade -p'. Same errors. Do I need to install packages from saucy-proposed before doing the 'do-release-upgrade'? Pl. advise. Thanks.

@Mike - Could you please add the /var/log/dist-upgrader/main.log file from your most recent upgrade attempt? Thanks in advance.

As requested.

@Mike - you are encountering bug 1241541 which you'd previously reported and not this particular one.

Got it. Sorry 'bout that.

This bug was fixed in the package ubuntu-release-upgrader - 1:0.205.2

---------------
ubuntu-release-upgrader (1:0.205.2) saucy-proposed; urgency=low

  * DistUpgrade/DistUpgradeCache.py: Do not initialize the cache until we know
    it is unlocked. (LP: #1214186)
 -- Brian Murray <email address hidden> Fri, 25 Oct 2013 10:01:55 -0700

The verification of this Stable Release Update has completed successfully and the package has now been released to -updates. Subsequently, the Ubuntu Stable Release Updates Team is being unsubscribed and will not receive messages about this bug report. In the event that you encounter a regression using the package from -updates please report a new bug using ubuntu-bug and tag the bug report regression-update so we can easily find any regresssions.